Ganymede An Extensible and Customizable Directory ...€¦ · Jonathan Abbey and Michael Mulvaney – The University of Texas at Austin ABSTRACT In the fall of 1994, Applied Research

The following paper was originally published in theProceedings of the Twelfth Systems Administration Conference (LISA ’98)

Boston, Massachusetts, December 6-11, 1998

For more information about USENIX Association contact:

1. Phone: 510 528-86492. FAX: 510 548-57383. Email: [email protected]. WWW URL: http://www.usenix.org

GanymedeAn Extensible and Customizable Directory Management Framework

Jonathan Abbey and Michael MulvaneyThe University of Texas at Austin

Ganymede: An Extensible and CustomizableDirectory Management Framework

Jonathan Abbey and Michael Mulvaney – The University of Texas at Austin

ABSTRACT

In the fall of 1994, Applied Research Laboratories, The University of Texas at Austin(ARL:UT) presented a paper [1] at LISA VIII, describing work that we had performeddesigning and implementing a management framework for NIS and DNS, called GASH. In theyears since that paper was presented, it has become clear that the design of GASH wasinsufficient to meet the complex, idiosyncratic, and rapidly changing needs of modernnetworking. GASH suffered from being too inflexible to be rapidly retooled for a changingnetwork environment, from being limited to a single user at a time, and from being unable toprovide management services to custom clients.

In the face of these issues, the Computer Science Division at ARL:UT went back to thedrawing board and developed a Java-based directory management framework on the basis of thedesign principles presented in our GASH paper. Written in Java, Ganymede 1 is based on adistributed object design using the Java Remote Method Invocation [2] protocol and features amulti-threaded, multi-user server, and a graphical, explorer-style client. By supportingcustomization through a graphical schema editor, plug-in Java classes, and external build scripts,Ganymede is able to support a variety of directory services, including NIS, DNS, LDAP, andeven NT user and group management.

Introduction

In early 1992, our laboratory had a problem. Wehad a need to pull computers from all over the labtogether into a common NIS [3] and DNS [4] regime,but the lab was separated into several roughlyautonomous groups. We needed a way to create a cen-tralized NIS and DNS domain while preserving theability of the groups to control their own user andgroup accounts. In addition, we needed to centralizeemail delivery and administer automounter volumedefinitions to support a useful and transparent networkarchitecture. In response to all this, the Computer Sci-ence Division at ARL:UT developed the GroupAdministration Shell (GASH, for short), a text-basedshell that allows designated users in our laboratory toissue commands that modify our NIS and DNS tables.We put this into operation in late 1993, and for the lastfour and a half years, we have run our laboratory onGASH, enjoying significant benefits in database con-sistency and simplicity of administration.

There have been problems with GASH, however.GASH was a very rigid program that directly manipu-lated NIS source files and a complex system databasefile that was transformed into DNS by a Perl script.Whenever we had a need to alter or elaborate anyaspect of our network computing environment, wefound that modifying GASH was extremely difficultand time-consuming. In addition, certain aspects of

1Which stands for The ‘‘GAsh Network Manager, DeluxeEdition,’’ of course.

GASH’s operation proved troublesome in practice.The permissions and ownership model used by GASHwas very idiosyncratic, and made it difficult to transferusers between groups. More fundamentally, GASHhad no clean way to interact with other tools. If a userwanted to change his password, he had to have hisGASH administrator change it with GASH, or useyppasswd and take the chance that the yppasswd dae-mon might conflict with an administrator makingchanges in GASH. GASH was clearly an inadequatetool to take us into the future. We wanted to be able totie our network and account management tools intoour personnel databases, we wanted to be able to mod-ify our network topology as needed without spendingsix months reworking the 50,000 lines of C code inGASH each time, we wanted to support LDAP andNT, and we wanted all of our end users to be able totake advantage of our management tools, whichwasn’t practical with a single-user tool.

By late 1995, we knew something had to bedone. Looking around, we were not able to find a suit-able and reasonably priced commercial tool that wasfocused on the issues we had developed GASH toaddress and that would give us the path to the futurewe wanted. The network management packages wewere aware of at the time were all focused on manag-ing distributed workstations rather than managing cen-tralized directory services.

We had a lot of experience and insights into theproblem domain we were dealing with. We knew wewanted a client-server system. We knew we wanted ageneric system that could be easily customized, and

1998 LISA XII – December 6-11, 1998 – Boston, MA 197

Ganymede: An Extensible and Customizable Abbey and Mulvaney

we knew we wanted a GUI. So, our task was set. Wewould work to build a GUI GASH Construction Set.

A GUI GASH Construction SetIn our LISA VIII paper [1], we observed that we

saw some potential in reworking GASH around anobject database [5] to facilitate automatic consistencymaintenance and the provisioning of a GUI. By late1995, a certain highly caffeinated object oriented pro-gramming language was making a lot of noise for itssophistication, ease of use, and portability. After doingan extensive design review, we determined that Javalooked as though it would enable us to meet ourdesign goals on all fronts. The Java Virtual Machineprovided us with the ability to have our code run onPC’s and Mac’s as well as on our UNIX workstationsand X terminals, and the Java Remote Method Invoca-tion (RMI) protocol allowed us to do a true distributedobject design [6] without having to worry aboutobtaining a CORBA ORB 2 for all the machines in ourlaboratory.

After spending the first half of 1996 doingdesign work with pen and paper, we began the work ofimplementing Ganymede. Two years and 140,000lines of Java later, in mid-1998, we are currently beta-testing the Ganymede system. We have produced arobust and customizable client/server system capableof doing everything GASH did, with plenty of room togrow. Our beta-testers have run the Ganymede serveron Sparc Solaris, AIX, Linux, and FreeBSD. Theclient has been run successfully on Windows 95 andNT using Sun’s Java browser plug-in and from thecommand line on the UNIX platforms mentionedabove. At the lab, we are running Ganymede on a testbasis with all the data from our installation of GASHloaded into the server and experiencing full function-ality and essentially perfect up time on the serverunder the 1.2 beta 4 JDK. We expect to have fullytransitioned to running the lab on Ganymede by thetime of the LISA 1998 conference.

What Is Ganymede?

Ganymede is a system for managing data whichis to be fed into a network through some standard dis-tribution mechanism, such as NIS, DNS, Rdist, LDAP,or the NT Domain Controller system. Ganymede isdesigned with an emphasis on providing tight controlover what types of changes can be made to thedatabase it manages, and on allowing multiple users tomake changes to that database simultaneously. The

2CORBA stands for the Common Object Request BrokerArchitecture. It is a standard for allowing object orientedcode to communicate, object-to-object, across networks. AnORB is an Object Request Broker, a piece of software thathandles the network communications on behalf of object ori-ented code on a given system. The CORBA specification isa product of the Object Management Group (OMG), andmore information can be found at their website [7].

Ganymede server is not intended to serve as a high-volume directory server, but rather is designed to workwith directory systems designed for high-volume use,such as NIS, DNS, and LDAP, whose servers may notthemselves provide useful mechanisms for managingchanges.

The Ganymede system is based on a client-serverdesign. The server contains a built-in object database,and supports custom Java plug-ins which provideintelligent management of object types defined in theserver. The server supports an admin console whichcan monitor the server and that includes a GUI schemaeditor that can alter the definition of the database heldin the server as the server runs. Several clients can betalking to the Ganymede server simultaneously, eachbrowsing the database, issuing queries, makingchanges, and committing transactions without interfer-ing with each other. Whenever a client commitschanges to the database, the server can schedule oneor more custom builder tasks to write out source filesfor NIS, DNS, or whatever is being supported, andthen run an external script to propagate the exporteddata into the network environment. See Figure 1 for adiagram of the overall system.

Ganymede Server

Client Client Client

ExternalBuildScripts

Plug-inSchemaCode

RMI

Transaction Manager Namespace Manager

Object Store

Permissions Manager Task Scheduler

Figure 1: Ganymede Block Diagram.

The Ganymede Server

The Ganymede server, like the rest of theGanymede system, is written entirely in Java. It con-tains over 100 classes, which provide for the storageand manipulation of objects, the management ofobject locking and transactions, the scheduling ofdatabase checks and external build processes, and acomprehensive ownership and permissions model,among other features.

Object StoreThe Ganymede server has a built-in object

database which is held in memory while theGanymede server runs. The objects in the database areheld in a set of thread-synchronized hashing datastructures. This design gives the server good perfor-mance and multi-threaded safety at the cost of a poten-tially large RAM footprint. The in-memory database isbacked by an on-disk ganymede.db file. During

198 1998 LISA XII – December 6-11, 1998 – Boston, MA

Abbey and Mulvaney Ganymede: An Extensible and Customizable

execution, the server dumps its database toganymede.db every two hours. Between dumps, theserver maintains a journal file, which is a record oftransactions made since the last database dump.

The Ganymede database is broken down byobject type. Everything held in the database is aninstance of a defined object type. Such object typesmight include things like ‘‘User,’’ ‘‘Group,’’ ‘‘Sys-tem,’’ and so on. There are a number of object typespre-defined in the database that the server depends onfor its operation. These are shown in Table 1, below.We will discuss all of these built-in object types inmore detail later. For now, notice that each object typein the database is identified by an object type code.The server can handle Object Type ID’s from 0 to 32k,but Type ID’s of 256 or less are reserved for theserver ’s internal use. Object Type ID’s above 256 areavailable for Ganymede adopters to define for theirown use.

Object Object Type IDOwner Group 0Admin Persona 1Role 2User 3System Event 4Builder Task 5Object Event 6

Table 1: Mandatory Object Types.

Each type of object in the Ganymede databasehas its own set of data fields defined. Each field has aname and ID, and can have certain options defined,depending on the type of field. The Ganymede serversupports eight different types of fields, as shown inTable 2. Most types of fields just hold a single value,but String, IP Address, and Object Reference fieldscan be defined to be vectors, holding up to 32k valuesin a single field. String, Integer, and IP Address fieldscan be connected to a namespace defined in theserver. The server manages such fields to make surethat the values in them are kept unique across the rele-vant objects and fields.

Field Type OptionsString Vector, Length, Chars AllowedInteger Max/Min ValuePassword Crypted/Non-CryptedDate Max/Min ValueBoolean Labeled/Non-LabeledPermission MatrixIP address Vector, IPv4 or IPv6Object Reference Vector, Target Type

Table 2: Field Types.

Most of these field types are self-explanatory,but a couple require some discussion. The permissionmatrix field type is used by the Ganymede

permissions system and is not really useful in anyother context. We’ll talk about where the server usesthe permission matrix field type when we discuss theGanymede permissions system. We do need to talkabout the object reference field type, but before weget into the details of this field type we need to talkabout how objects in the server are identified.

Invids and Invid Fields

Objects in the database are identified by anobject called an invid, which stands for INVariant ID.An invid is a Ganymede object identifier, and isimplemented as a pair of numbers. The first number isthe object’s type id, the second is a number between 0and 2 billion unique to that object within its objecttype. Object numbers are never re-used. This makes itpossible to unambiguously track the history of anobject in the server’s logs, but it does limit the serverto handling 2 billion objects of a particular type overits lifetime.

The object reference field type is simply a fieldthat holds invids. In fact, from now on, we’ll refer tothis field type as an invid field. Invid fields are usedthroughout Ganymede to link objects together. Whenone object’s invid is placed in an invid field in anotherobject, those objects are said to be linked. All objectlinks in Ganymede are symmetrical, so that eachobject has references to all objects in the database thatpoint at it, and vice versa. Because all objects in thedatabase are symmetrically linked, the database caneasily be kept up-to-date whenever objects are deleted.All that the server has to do in order to clean up afterdeleting an object is to modify all objects that werelisted in the invid fields of the deleted object; it is notnecessary to sweep through the entire database look-ing for linked objects. Another advantage of usinginvids to link objects is that objects in the database canbe renamed or relabeled without disturbing the link-ages established in the server.

Invid fields can be configured so that an invidfield in one object is linked to an invid field in anotherobject. This is shown in the schema editor screen shotin Figure 2. In this screen shot, we see the users fieldin the group object being edited. The users field in thegroup object is an invid field that points to the groupsfield in the user object. When the client edits a groupobject, the server will automatically provide a list ofusers that can be placed in this field. Adding a user tothis group will automatically cause the group to beadded to the user, and vice versa. The user can look ateither object and see the relationship. This bi-direc-tional linking is very important to the structure of theGanymede server. It is responsible for a lot of theintelligence of the server. If a user were to try to deletea group, but didn’t have permission to edit the userslisted in the group, the server would detect this andmight reject the operation, depending on how theschema was configured.



Some object reference fields are ‘‘edit-in-place,’’which means that the objects referenced by that fieldare handled as if they were contained within the refer-encing object. An object type must be designated inthe schema editor as an embedded object in order tobe linked to an edit-in-place field. An embeddedobject is for the most part very much like any otherobject in the database, but it does not have its owner-ship and permissions tracked independently of its par-ent, and the server handles its creation and deletion alittle bit differently. The easiest way to see the differ-ences between embedded and top-level objects is bylooking at the special fields the server uses to keeptrack of these objects.

Figure 2: The Schema Editor Editing an Invid Field.

Mandatory Fields

Just as there are mandatory object types, so tooare there a number of mandatory field definitions. Thefields shown in Table 3 are defined in all non-embed-ded object types in the server. All field ID’s below 100are reserved for global fields (fields defined in all top-level objects). Field ID’s between 100 and 256 aredevoted to fields in the mandatory object types that theserver depends on for its operations. Field ID’s above256 are user-assignable fields and can be configured inthe schema editor.

Embedded objects have a different set of manda-tory fields, which are shown in Table 4. The con-tainer field is simply a pointer to the object that theembedded object is contained in. This field is linked tothe field in the parent where the embedded objectappears.

Field Type Field IDOwner List Invid Vector 0Expiration Date Date 1Removal Date Date 2Notes String 3Creation Date Date 4Creator Identifier String 5Last Modification Date Date 6Last Modifier Identifier String 7Back Links Invid Vector 8

Table 3: Mandatory Fields For Top-Level Objects.

Field Type Field IDContainer Invid 0Back Links Invid Vector 8

Table 4: Mandatory Fields For Embedded Objects.

Most of the mandatory fields should need noexplanation. We’ll discuss the owner list field whenwe talk about the Ganymede permissions system a bitlater on. For now, let’s talk about the back links field.

The back links field is a bit special. Previously,we said that the server guarantees that all referencesmade in an invid field are symmetrical, and we gavethe example of user and group object types havingtheir groups and users fields symmetrically linked.This kind of visible bi-directional linking sometimesdoesn’t make sense. In cases where it doesn’t, an invidfield can be configured so that it points to an objectwithout specifying the target field. The server will use



the back links field to handle the back-reference. Theclient won’t see this back-link, but the server will, andwill use it when the referenced object is deleted.

Figure 3: The Schema Editor Binding a Custom Class.

package arlut.csd.ganymede.custom;

import arlut.csd.ganymede.*;

public class userCustom extends DBEditObject {

// Boilerplate Constructors Omitted

public boolean fieldRequired(DBObject object,short fieldid)

{switch (fieldid){

case userSchema.USERNAME:case userSchema.UID:case userSchema.HOMEDIR:case userSchema.LOGINSHELL:

return true;case userSchema.PASSWORD:

return !object.isInactivated();}

return false;}

}

Figure 4: A Simple Custom DBEditObject Subclass.

The DBEditObject Class and Server CustomizationOne of the keys to the Ganymede server’s flexi-

bility is that it takes advantage of Java’s object ori-ented language features and dynamic linking to allowindividual customizers to write classes to manageobjects. The DBEditObject class in the server is con-sulted on every major decision having to do with howthe server should handle operations on an object. TheGanymede schema editor allows adopters to bind

custom DBEditObject subclasses with object types inthe server. Figure 3 shows the arlut.csd.ganymede.custom.userCustom class being bound to the userobject.

DBEditObject provides over two dozen methodsthat can be overridden by custom logic to inject intelli-gence into the Ganymede server. While it is out of thescope of this paper to describe all of the ways in whichDBEditObject can be customized, we can mention afew highlights, and we will provide a simple sampleof customization through DBEditObject subclassing.



The DBEditObject class provides the ability forcustom code to extend or override the default permis-sions system, to approve or deny any change to fieldswithin an object based on the contents of the object, itsrelations with other objects, or the identity of theadmin seeking to make the changes. It can provide alist of valid choices for string and invid fields. It canreturn custom dialogs in response to attempted opera-tions, or even involve the client in a step-by-step wiz-ard interaction sequence. It can get involved when atransaction is committed, to take actions outside of thedatabase, such as creating home directories whenusers are created, or connecting changes to an objectin the Ganymede database to an external database.

Figure 6: Ganymede Admin Persona.

For a simple example of what is involved in aDBEditObject subclass, see Figure 4. This exampleshows the code necessary to specify what fields mustbe present in a user object when a transaction is com-mitted.

Permissions and OwnershipOne of the critical elements of Ganymede’s

design is the permissions model. Ganymede providesa universal permissions model that allows completeflexibility in apportioning privileges to classes ofusers/administrators, without becoming so unwieldyas to be impractical. The model is designed to supportgroup administration, with ownership over objectsshared by groups of administrators. Different classesof administrators can be defined, each with differentprivileges over different kinds of objects, and differentfields within those objects. The Ganymede server hasthree object types defined to support this model, theadmin persona, owner group, and role objects, asshown in Figure 5.

Each user in the Ganymede system can have oneor more admin personae associated with it. The adminpersonae represent administrative privilege sets thatthe user can select, through an su-like mechanism in



the client. An admin persona object marries a list ofowner group objects and a list of role objects to definethe objects and operations that can be performed. Theowner group objects define what objects are consid-ered to be under the ownership of that admin persona,while the role objects define what operations theadmin persona is permitted to perform.

Figure 7: Ganymede Owner Group.

ObjectsIn

DatabaseAdmin

Persona

AdminPersona

AdminPersona

Owner Group

Owner Group

Role Role

User

Default Role

User

Figure 5: Ganymede Permissions Objects.

Admin Personae

All users registered with Ganymede may havesome minimal permissions granted them by thedefault role object. This will typically include theability to edit their own passwords, shells, and fingerinformation, and to view some information aboutother users registered in the Ganymede database. If anindividual is to have more privileges than that, anadmin persona must be created for the user, as shownin Figure 6.

An admin persona includes a password, whichmust be entered by the user in order to access theirextended privileges, a list of owner groups and roles,check boxes indicating whether the admin is privi-leged to run the server-monitoring console, and anoptional email address for Ganymede to use to sendmail to in response to the admin’s actions.

Owner Groups

As mentioned above, all objects in the databaseare owned by owner groups, rather than by individualadministrators. This design decision came out of ourexperience with GASH. By having all ownershipvested in owner groups rather than in individual



administrators, it is possible to bring a new adminis-trator into a group without having to manually add thatadministrator to the ownership list of hundreds orthousands of objects. Other fields defined in the ownergroup object (see Figure 7) are designed to let admin-istrators in a group share email notification for actionstaken on objects owned by that owner group.

Figure 9: System Netgroup Objects Owned by an Owner Group.

The Ganymede server supports a hierarchy ofowner groups. One owner group can own another. Notonly do the admins in the first owner group have own-ership rights over the second, but also over all objectsowned by that owner group. Figure 8 illustrates this.Admins belonging to the engineering owner group inFigure 8 own not only the hardware and softwareowner groups, but also all of the objects owned bythose groups. In addition, owner groups are consid-ered to own themselves, so an admin belonging to theengineering owner group could, if permitted by theroles granted to him, add or delete admins from theengineering group as well as the hardware and soft-ware groups.

The supergash owner group is special; allobjects in the database, including all owner groups,are implicitly owned by the supergash owner group.Any admins belonging to the supergash owner grouphave ‘‘root’’ privileges over the Ganymede database.

supergash

Software Division

Engineering Group

Hardware Division

Figure 8: A Hierarchy of Ganymede Owner Groups.



An admin with editing privileges for an ownergroup can add or remove objects from that ownergroup by simply editing the owner group. This isshown in Figure 9.

Figure 10: A Ganymede Role.

Roles

The role object defines what an admin personacan do, both for objects owned by the admin throughhis owner group membership, and for objects ‘‘atlarge’’ in the database. Each role object contains twopermissions matrices: one for objects owned, andone for default permissions. These permission matri-ces contain an array of booleans which allow access tothe database by object type and field, with create, edit,view, and delete permissions categories. An admin canonly view, edit, create or delete objects and fields thatare specifically granted him by his set of roles. SeeFigure 10 for a list of fields defined in the role object.

Of special interest in Figure 10 is the ‘‘Delegat-able Role?’’ check box. In order to support a truehierarchy of administrative control, admins can begranted the power to create new roles, and to createnew admins. An admin who creates a new role oradmin may not grant that role or admin privileges thathe himself does not have from a delegatable role. That

is, if an admin has the GASH admin and secretaryroles, and only the secretary role is delegatable, theadmin will only be able to grant the secretary role toadmins that he creates, and if he creates a new Role,will only be able to set bits in that Role that he gotfrom either the secretary or default roles. Figure 11demonstrates this. The check boxes that are visiblecorrespond to bits that the admin editing the role hashimself had granted to him through a delegatable role.The boxes X’ed out represent privileges that thisadmin may not grant to other roles.

The combination of the owner groups, whichdetermine which objects are accessible, and the roles,which determine what can be done to those objects,provides complete flexibility while maintaining theability to make wide-ranging changes in the authoriza-tion schema by simply editing one or two objects inthe Ganymede database.

As implied above, all of the objects in theGanymede server, including the owner group, role,and admin persona objects, are administered by thispermissions system. The same permissions system thatcontrols access to the Ganymede database also con-trols access to the controls themselves.



As powerful as this system is, it is not complete.There will be cases where a more specialized permis-sions model is required. Take for instance the case ofmaintaining a public mailing list where users shouldbe able to add and remove themselves, but not touchany other user in the list. All that would be required tosupport this sort of model would be to bind a customDBEditObject subclass to the Mail List object type inthe Ganymede server and redefine the anony-mousLinkOK() and anonymousUnlinkOK() meth-ods in DBEditObject. Other methods in the DBEditO-bject class can be overridden to implement customownership determination logic, or even to entirelyoverride the normal persona/role/owner group permis-sions system.

Figure 11: Permission Bits in a Role.

TransactionsThe Ganymede server is built around a transac-

tional model wherein clients connected to the servercheck out objects for editing. There may be manyclients connected to the server simultaneously, butchanges made to objects in one transaction will not bevisible to other users until the transaction is commit-ted. Queries issued by clients are guaranteed to beatomic with respect to transactions across the durationof their processing.

The server supports checkpointing and rollbackwithin the course of a transaction. This allows forcomplex sequences of operations to be attempted andundone if the sequence could not be carried to comple-tion successfully.

The transaction commit process in theGanymede server is based on two-phase commit logic,so that custom code can be written to connect

transactions issued in Ganymede to transactions inexternal databases.

In the Ganymede server, whenever transactionsare committed, a record of the transaction is written toa journal file. This journal file allows the server torecover any transactions that were committed betweenthe time that the server last performed a full databasedump and an abnormal shutdown. The worst case forthe Ganymede server on power failure or server crashis the loss of the most recently issued transaction.

When transactions are committed, the Ganymedescheduler schedules external build processes for exe-cution. If multiple transactions are committed whilethe Ganymede scheduler is still executing the previousexternal build, the Ganymede scheduler will simplyinitiate another external build when the first buildcompletes. Thus, multiple transactions made by usersmay be propagated to the external environment inbulk, depending on the rate that transactions are com-mitted and the time necessary to complete an externalbuild.

Logging and Email NotificationGanymede, like GASH before it, has a very thor-

ough logging and email notification system. All sig-nificant events on the server are logged to disk, andcan also be emailed to interested parties. There are awide variety of system events built into the server,and a supergash-level administrator can customize theserver ’s email notification behavior. Figure 12 showsthe notification options for a system event.

While the system event categories are more-or-less hard-coded into the server, it is important that cus-tom object types can have their significant events



handled specially as needed. To support this, theGanymede server also supports object events, whichmeld one of a set of common happenings to objectsalong with the type of object of interest. Figure 13tells the story.

Figure 12: A System Event Record.

One of the lessons we learned from GASH is thatit is often important for admins to be able to tell whathas happened to objects under their control. Inresponse to this, the Ganymede log file has beendesigned to be easily parsed. The server itself can scanits own log file to report on changes made to anobject, as shown in Figure 14.

Background TasksThe Ganymede server includes a built-in task

scheduling facility, similar to cron. A backgroundthread queues various tasks for execution, includingthose shown in Figure 15.

The ‘‘garbage collection’’ task runs every nightat midnight to do a bit of preemptive house cleaning inthe server. The ‘‘database dumper’’ task runs everytwo hours, consolidating the database and cleaning thejournal file. The ‘‘expiration’’ and ‘‘warning tasks’’each run once a day, to handle objects that have had

their expiration or removal times set. The warningtask is responsible for looking at objects that willexpire or be removed in the near future, and sendingout warnings to the administrators responsible forthose objects through email. And then, there are thebuilder tasks.

Builder Tasks

Just as it is possible to define custom subclassesof DBEditObject to provide custom management ofobject types in the server, it is also possible to definecustom builder tasks to be loaded into the server atruntime, as in Figure 16. Whenever a transaction iscommitted, any builder tasks registered with the serverare scheduled for execution. The builder tasks willscan the Ganymede database, write out source files forNIS, DNS, etc., and call an external shell script to takethose source files and propagate the data into NIS,DNS, or whatever else is being managed.

Schema Kits

The combination of a Ganymede schema defini-tion, a set of custom DBEditObject subclasses, andany custom builder tasks together comprise aGanymede schema kit. As currently available for



download, the Ganymede server includes threeschema kits. One is an ‘‘nisonly’’ kit that handlesSolaris-style passwd and group files, another is a‘‘bsd’’ kit for managing BSD 4.4 master.passwd andgroup files, and the third is a full-scale ‘‘gash’’ kit thatwas designed as a way for adopters of GASH to havea minimal-work ‘‘drop-in’’ replacement for their exist-ing GASH installations. Each of these kits includes aloader program to scan the original files and create aganymede.db file which Ganymede can then manage.

Figure 13: An Object Event Record.

The nisonly and bsd kits are provided so thatUNIX admins can download Ganymede, load theirpasswd and group files, and start playing withGanymede without a large commitment of time. TheGASH kit is a much richer environment, and imple-ments the complex network management logicdescribed in our LISA VIII paper [1]. The GASH kit,like GASH itself, is designed to manage a single DNSdomain and a single NIS domain, with support forNIS-specific features like netgroups and automounterconfiguration maps.

We are working on developing a next-generationschema based on the GASH model. This schema is tohave richer support for DNS, including support forgeneric subnetwork allocation. We also want to sepa-rate out personal identification from the user account.Having separate person objects would facilitate propergeneration of a canonical LDAP directory for our

laboratory, and would allow us to track responsibilityfor user accounts more conveniently.

Another design feature of our next-generationschema is support for NT and UNIX integration. Weare currently shadowing our UNIX accounts into ourNT domain controller using Rsh and some Perl scriptson the NT side. In our next-generation schema weplan to have a check-box on user objects to controlwhether or not the account should be replicated on NT,and to support NT group accounts as well as UNIXgroup accounts within Ganymede.

There are quite a number of other interestingpossibilities for schema development. One obviouspossibility is a DNS-only schema, with support formanaging a large number of DNS domains and IPaddress ranges. We have demonstrated in our workimplementing the GASH kit in Ganymede that theDBEditObject customization hooks are adequate tohandle multiple-interface systems and network alloca-tion. Creating a DNS-only schema kit would be somework, but we do not believe that the task would beoverly difficult, and such a schema kit could be ofconsiderable utility for ISP companies.

The Ganymede Client

The Ganymede server supports a generic Javainterface for clients, allowing various custom clientsand automated processes to talk to the server. After



connecting, clients can manipulate objects throughRMI references to the objects in the database.

Figure 14: A User Object’s Sordid History.

The generic client API permits us to write a vari-ety of custom clients to handle specific tasks. Forexample, we have developed a command-line-basedapplication that changes a user’s password and looksjust like Unix’s passwd command. After prompting forthe user’s current and new passwords, this client logson to the server, checks out the appropriate userobject, changes the password, and logs out. Clientscould easily be written to do bulk edits to the database,or to tie automated processes of various sorts to theGanymede server.

Most of our effort in client development hasgone into developing a generic GUI client. This clienthas very little customization built in to it; it queries theserver at run-time to get information about the schemabeing used. With a few exceptions to provide specialhandling for the mandatory object types that theGanymede server depends on, it is completely generic,and can be used with any database definition in theserver.

This primary Ganymede client displays a largewindow with two main panels. The left panel displaysa tree that lists all the objects in the database, sortedby object type, while the right panel holds windowsused for viewing and editing objects. The tree is builtwhen a user first logs in, and can be used to browsethe database. Each node of the tree has a context-sen-sitive menu associated with it, which is accessible byright-clicking on the node.

When the client first connects to the server, itopens a new transaction. The user can commit or can-cel changes to the database made during the opentransaction by using the ‘‘Commit’’ or ‘‘Cancel’’ but-ton in the lower right-hand corner of the client. Nochanges are made to the database until the ‘‘Commit’’button is clicked, so the user has the ability to cancelany actions he performed during the transaction.

Logging In

The client can be run either as an applet in a Webbrowser or as an application. When the Ganymedeclient is run in a Web browser, it appears as an appletin the browser’s window, as shown in Figure 17.After entering a correct username and password, the



main client window appears. At this time, the clienthas queried the server to determine what sort ofobjects have been defined and to retrieve a definitionfor each type of object that the user will be able toedit. The client caches this information to speed uplater operations.

Figure 15: The Admin Console, Showing Registered Tasks.

When logging in as an end user, the only visibleobject in the tree is the user’s own user object. In orderto gain more privileges, a user can access his adminpersona by choosing a persona on the Persona menu.The user will be presented with the dialog shown inFigure 18, where he will enter the password for hisadmin persona. Successfully changing to a new adminpersona causes the client to rebuild the tree in order toreflect the expanded permissions, as well as to closeany windows that might be open.

By default, the tree only shows those objectswhich are editable by the current admin persona, butthe user can choose to have the tree display all theobjects he has permission to view by selecting theShow All Objects menu item from the menu shown inFigure 19. Typically, the default role will grant userspermission to view only a small number of objects,

such as other users in the same group. Most adminis-trators will be have roles assigned them which grantprivilege to browse more of the database.

Editing Objects

After opening an object node, a list of theeditable objects of that type is shown. When the ‘‘EditObject’’ menu item is chosen from the tree’s pop-upmenu, an editable window is placed in the right-handside panel. This window allows for the editing of thisobject.

When an object is first opened for editing orviewing, the server generates and sends the client aremote reference corresponding to that object. Theclient then talks directly to the object to determinewhat fields are defined within it, and builds up thefields displayed in the editing window.

Most types of fields in an object are displayedwith simple GUI widgets: check boxes for booleanfields, date fields for dates, and text fields for singlestrings, IP addresses, and numbers. Object referencefields are either pull-down lists for scalar fields, or acomposite selector for vector lists. The selector showsa list of available references on one side, and a list of



selected references on the other. By moving the labelsbetween the two lists, references are added or removedfrom the field.

Figure 16: Registering a Builder Task.

Figure 17: Ganymede Login Screen.

Embedded objects are handled by embedding anedit panel within the main edit panel, as shown in

Figure 20. The embedded object’s fields can be hid-den or revealed by clicking on a icon in the panel’scontainer. Because embedded objects are alwaysemployed in a vector context, there are also controls toadd or delete objects of the appropriate type from thecontaining object. Embedded objects may themselves



contain more embedded objects, and so a complexhierarchy of containment can be managed, if neces-sary.

Figure 18: Admin Password Dialog.

Figure 19: Object Menu.

Return Values

When a field is changed, the server reports to theclient by sending a special object called a returnvalue, which tells the client what happened as a resultof the change. In a simple case, the return value

contains information about the success or failure ofthe change. The return value is capable of muchmore, however; it can instruct the client to display aninformational dialog, and it can tell the client whichother fields need to be rescanned because of thechange to the current field, either in the current objector other open objects. This causes the client to querythe server and update the specified fields immediately.Also, the server can use the dialog to initiate a



complex series of wizards to gather more informationabout a complicated action.

Figure 20: Editing an Embedded Object.

Figure 21: Picking a New Home Group.

Sometimes changes to the database require moreinformation than can be provided by simply editing a

single field. In such cases, the server can send a defi-nition for a custom dialog to the client. The dialogmay include graphics, text, and a range of GUI fields.The information from this dialog is sent back to the



server, which may in turn send down a new dialog tocontinue the discussion. In this way the server canwalk the client through a series of steps using the tra-ditional GUI wizard.

Figure 22: The Query Box.

Figure 23: The Query Result.

For example, when editing a group, removing auser reference from the home users field de-selectsthat group as the user’s home group. This is demon-strated in Figure 21. In the GASH Schema, a usermust have a home group, so a new home group mustbe chosen. In order to find out which group the userobject should now have as the home group, the serversends a dialog with a list of the user’s current groupsto the client.3 After choosing a group from this list theserver places the user in the new home group, andcompletes the operation initiated when the user tried

3Actually, the server only does this if the user belongs totwo or more other groups. If the user only belongs to onegroup, that group is designated as the home group. If the us-er does not belong to any other groups, the operation willnot be permitted. All of this logic is coded into the GASHschema, which we are using for demonstration purposes inthis paper.

to modify the home users field.

Queries

The Ganymede server provides a powerful andflexible query mechanism. The client uses this systembehind the scenes in many places. The client uses theserver ’s generic query mechanism when it loadsobjects into the tree, for example. Users can createtheir own queries as well, using the query box shownin Figure 22.

The query box allows the user to build rathercomplex queries. The query box is interactive, anduses the client’s cache of schema information to guidethe user as to what fields and choices are available.Multiple query terms can be put together to specify asnarrow a search as is desired. The user can furthercustomize the query by indicating what fields shouldbe returned in the server’s report. When the query issubmitted, the results are displayed in a table, like theone in Figure 23. The table contains one row for eachobject matching the query, and displays the values ofthe fields in the columns of the table. The table allowsfor sorting on each column, rearranging column



widths, and exporting the contents of the table eitherto disk or as an email message. Reports can be gener-ated, either as tab- or comma-separated ASCII, orencoded as an HTML table.

Adoption Considerations

We hope that Ganymede will prove attractive toa large community of administrators. We feel that itssophistication, flexibility, and portability will allow itto be widely adopted in a way that GASH could notbe. Ganymede is not appropriate for all environments,however. There are issues regarding security and per-formance that should be taken into account when con-sidering Ganymede’s suitability.

SecurityWhile Ganymede has an effective permissions

and ownership model, it does not support encryptedcommunications between the client and the server. Atthe current time, Ganymede is based on RMI over anon-encrypted socket layer, and as such is not appro-priate for use on the open Internet for those who areconcerned about packet-sniffing.

Sun is moving to support RMI over customsocket classes in the Java 1.2 JDK, which will make itpossible to produce a version of Ganymede that usesSecure Socket Layer (SSL) encryption. Sun has notyet committed to producing a freely available imple-mentation of SSL sockets for Java, but third-partycompanies have demonstrated RMI over SSL usingtheir SSL implementations [8]. These implementationstend to be very pricey, however, and there are theinevitable U.S. export and licensing issues to consider.

When and if an affordable SSL solution for RMIappears, Ganymede could be retrofitted to provide ahigher level of security with very little work.

Performance and ScalabilityWhen we originally designed Ganymede, we set

as a goal the ability for Ganymede to scale up to han-dle 50,000 network object records, a figure some sixtimes higher than we were handling with GASH. Themain factors determining how well Ganymede willscale are memory, Java Virtual Machine (JVM) effi-ciency, and thread and database contention in theGanymede server.

Memory

Ganymede has limitations on its scalability dueto its RAM-resident database. Our current database isloaded with DNS information for 2,087 systems with2176 total interfaces, 48 administrators, 753 useraccounts, 238 account groups, 1200 email aliasrecords, 234 NIS netgroups, 520 NFS volume defini-tions, 753 automounter entries for home directories,and network connectivity information for 517 rooms.This comprises a total of about a megabyte of GASHdata files in text form, some 8,526 objects or so in theGanymede server.

With this data loaded, the Ganymede server takesup just under 40 megabytes of RAM. Since themachine we are running the Ganymede server on has agigabyte of memory, scaling up by a factor of six or sowould not be a problem for us, at least as far as abso-lute RAM consumption is concerned.

JVM Efficiency

The biggest problem with such scaling is not theactual consumption of memory, but the time requiredfor the server to handle garbage collection. In an ear-lier stage of development, the Ganymede server wastaking up nearly 100 megabytes of RAM with ourdataset loaded, and we would experience frequent andsignificant lags in the server’s responsiveness. Wewere, however, running the Ganymede server on a rel-atively slow, relatively loaded machine, a 60-MHzmultiprocessor Sparc system, which was also respon-sible for the laboratory’s Web, news, mail and file ser-vices. In addition, we were running the JVM in inter-preted mode with debug logging enabled, rather thanusing the JVM’s Just-In-Time compiler (JIT) to con-vert the Java code to Sparc machine code. The latest1.2 beta seems to be much more efficient, however.We are expecting that a good generational garbagecollector, such as is promised with Sun’s next genera-tion HotSpot JVM, will do away with concerns overgarbage collection overhead, even with a substantiallylarger database.

The biggest delays that users are likely toencounter will occur when the builder tasks are run-ning. The builder task base class is designed so thatthe builder task can lock the database while it assem-bles the data needed for the build, and then release itwhile it is running external scripts to process the data.While the first phase of the builder task is running, notransactions may proceed to commit. A commonsource of delay is for a user to make some changes,commit the transaction, then immediately try to makesome more changes and commit that transaction. Thesecond transaction cannot proceed to completionwhile the server is busy writing out data files inresponse to the first.

Thread and Database Contention

Which brings us to the issue of thread anddatabase contention. The Ganymede server has beendesigned to try to keep thread contention low. Querieson the Ganymede database are a very frequent occur-rence, and the query system has been specially opti-mized. Any number of clients can issue queries on thedatabase without experiencing thread contention foraccess to the objects in the database, as long as notransactions are committing. While a transaction isbeing committed, no queries can be issued on objecttypes involved in the transaction until the transactionfinishes committing. Individual object accesses pro-ceed normally, as the server does do table-level syn-chronization at all times. Queries are guaranteed to be



transaction-consistent, and are not processed while atransaction is being committed.

One important thing to note with regards to scal-ability is that the Ganymede server does not supportmultiple users simultaneously making changes to anindividual object. This is most important with ownergroups. If two administrators in the same owner grouptry to create an object and place it in that owner group,one of them will be unable to do so, and must waituntil the other either commits or aborts his transactionso that the owner group is released. Whenever anobject is added to or removed from an owner group,such contention can arise. Simply editing an objectowned by a particular owner group, however, will notcause such contention.

The answer to this contention problem is to takeadvantage of the ability of the Ganymede server tosupport owner group hierarchies. If a group of adminsgets large enough that the admins are often getting ineach other’s way trying to create or delete objects, it isa simple thing to create sub-owner groups that theobjects can be placed in without contention.

Current Performance

With the current 1.2 beta 4 JDK we have had fiveusers and an admin console connected to theGanymede server simultaneously, with each user mak-ing changes and browsing the database without notice-able pauses. We do look forward to movingGanymede to a modern UltraSparc server at somepoint, but even at the current level of performance,Ganymede provides very acceptable performance forour needs. Right now, we have less than 50 adminis-trators registered in GASH, so it is rather unlikely thatmore than five admins will ever be using Ganymede atthe same time. Once we have fully replaced GASHwith Ganymede in the lab, we will be looking to seehow well Ganymede scales. Ultimately, we hope toallow our end-users to have free access to Ganymedeto handle their own passwords, shell information, andthe like. At this time we don’t know how heavilyGanymede will wind up being loaded in that environ-ment, but it seems unlikely that too great a number ofusers would ever try to change their passwords at thesame time.

Reflections on Java

When we first started investigating the possibil-ity of developing a next-generation GASH, Java wasfar less developed than it is today. Version 1.0 was theexciting new thing, and critical features like RMI hadnot yet been released. But the promise of a sophisti-cated, operating-system agnostic development plat-form with widespread industry support was com-pelling. When Sun released RMI Alpha2 in mid-1996,it became clear that a distributed object design forGanymede would be possible with Java. At that point,the basic outline of the Ganymede design started totake shape.

Since then, we have developed Ganymede asJava has itself been under development. In some caseswe have wound up developing pieces that were not yetavailable from Sun, such as the tree and table compo-nents used in the client and the admin console. Inmany other cases, such as with RMI, we have foundSun providing just the right thing at the right time tomake our work possible. We’ve had to wrestle withfrustrating bugs as Java has matured, but the bugs gotfixed. The momentum behind Java, both from Sun andfrom other companies has continually reinforced theappropriateness of our initial decision to go with Java.

Indeed, we do not believe that we could havedone Ganymede with any other technology, given theresource constraints we were under. Java gave us aportable GUI, a distributed object API, a large set ofthread-safe class libraries, and both memory and typesafety. Most of these things can be had for C or C++,but only for serious money and/or limited portability.We certainly couldn’t think of making a freely dis-tributable tool using commercial C++ class libraries.With Java, we were able to write a distributed, multi-threaded, portable GUI application of 140,000 lines ofcode using nothing but X-Emacs and the Java Devel-opment Kit.

Concluding Thoughts

Ganymede represents an attempt to provide acomprehensive and flexible management system thatcan be placed on top of the existing directory infras-tructure already in place in typical UNIX networks. Itis not designed to answer all the questions about howto organize directory services, but rather to allowadopters to bring their own experience and environ-ment into the design of their directory managementtool box. In this respect it differs greatly from GASH,which had a particular network management philoso-phy hard-wired into its implementation.

Ganymede seems most similar to Novell’s Nov-ell Directory Services [9] and Microsoft’s forthcomingActive Directory [10] in as much as it provides both acustomizable directory database and a set of GUI toolsto manage the database. It differs from these in that itdoes less; the Ganymede server is not designed to actas a high volume directory server. Ganymede does notsupport database replication or distributed manage-ment with multiple Ganymede servers. Instead it isdesigned to leverage existing directory mechanismssuch as NIS, DNS, and commercial LDAP serverswhich have their own mechanisms for providing relia-bility and scalability through backup servers.Ganymede depends on the flexibility of scriptingmechanisms on UNIX to provide support for gettingthe directory data where it needs to go. Finally,Ganymede does not provide native support for LDAPor, indeed, any other standard directory API.Ganymede can feed data to servers which supportsuch API’s, but administrative programs written to



manage directory services using such API’s will findGanymede, on the whole, an incompatible partner.

Ganymede does, however, provide a good solu-tion for the medium-to-large-sized UNIX or mixednetwork. It has particularly good support for groupadministration, with the permissions system and themail and logging system designed to facilitate admin-istration teams. In addition, the Ganymede server iscustomizable in a rather deep way. Not only can thedefinition of the database schema be customized, butalso a considerable amount of intelligence can beplaced in the server using plug-in Java classes. Thisnotion of an intelligent server is in keeping with ouroriginal design goal for GASH to produce a tool thatwould make it possible for a broad audience to safelymanipulate our centralized directory information.Ganymede can be taught about a particular environ-ment, and will work to keep it in good order.

Anticipated Future Developments

Barring some continuing polishing work, theGanymede system is complete and ready to use as it istoday. Our main goals at this point are to getGanymede fully implemented within our laboratory,and to get Ganymede documented well enough thatpeople can begin to work on developing their owncustom schema kits. Essentially, most things that wecan see needing to be done with Ganymede revolvearound the development and elaboration of schemakits.

One important development would be to imple-ment secure authentication and encryption for client-server communications. As we mentioned in our secu-rity discussion above, this will depend on an afford-able implementation of the SSL protocol for Java.

One possible downside of Ganymede comparedto GASH is that the Ganymede client requires a GUIdisplay, whereas GASH could be run from a TTY con-sole. It would be nice to have a completely functionaland generic text client for Ganymede for those caseswhen a GUI display is not available.

Availability

Ganymede is currently available in pre-release atftp://ftp.arlut.utexas.edu/pub/ganymede/, with docu-mentation, screen shots, and information on joiningthe Ganymede developer’s mailing list athttp://www.arlut.utexas.edu/gash2/. While we don’tyet have the licensing finalized, we intend to placeGanymede under the GNU Public License. Once wehave Ganymede fully implemented in the laboratoryand have licensing approved, we will formally releaseGanymede 1.0.

Credits

Many people have contributed to the develop-ment of Ganymede. In addition to Jon and Mike,Navin Manohar and Erik Grostic made important

contributions to the client-side code development. GilKloepfer provided design guidance for handling net-work issues and code for the GASH kit’s back-endDNS support. Dan Scott supported the project admin-istratively, and contributed greatly to the higher leveldesign issues in Ganymede, as well as providinginvaluable user interface design feedback and bugreporting.

A lot of Ganymede is based on the experienceand design work that went into GASH. In addition tothe aforementioned names, Dean Kennedy and PugBainter should be credited for their design work onGASH. Pug Bainter authored the original GASHmakefiles that Ganymede’s GASH kit uses to propa-gate information from Ganymede into NIS and DNS.

Finally, thanks to all of the administrators atARL:UT and elsewhere who provided feedback onGASH and let it be known that things could perhapsbe just a little bit better.

Author Information

Jonathan Abbey initiated the Ganymede projectat ARL:UT in late 1995 and has been working on itpretty much nonstop ever since. He graduated with aB.S. in Computer Science in 1993 from The Univer-sity of Texas at Austin, and has worked at AppliedResearch Laboratories since September 1989. He isreachable at [email protected] .

Michael Mulvaney has worked on the Ganymedeproject since joining ARL:UT in early 1997, develop-ing the Ganymede client. He graduated with a B.A. inEconomics in 1996 from The University of Texas atAustin. He is reachable at [email protected] .

References

[1] Abbey, J. ‘‘The Group Administration Shell andthe GASH Network Computing Environment,’’Proc. LISA VIII, September, 1994.

[2] Sun Microsystems, http://java.sun.com/products/jdk/rmi/ .

[3] Stern, H. Managing NFS and NIS, O’Reilly &Associates, Inc., 1991.

[4] Albitz, P., Liu, C. DNS and BIND, O’Reilly &Associates, Inc., 1993.

[5] Loomis, Mary E. S., Object Databases: TheEssentials, Addison Wesley, 1995.

[6] Hogan, C., Cox, A., Hunter, T. ‘‘DecentralisingDistributed Systems Administration,’’ Proc.LISA IX, September, 1995.

[7] Object Management Group, http://www.omg.org/ .

[8] http://www.java.sun.com/products/jdk/1.2/docs/guide/rmi/SSLInfo.html .

[9] Novell Corp., http://www.novell.com/products/nds/index.html .

[10] Microsoft Corp., http://www.microsoft.com/ntserver/basics/future/activedirectory/ .




Documents

Ganymede An Extensible and Customizable Directory ...€¦ · Jonathan Abbey and Michael Mulvaney – The University of Texas at Austin ABSTRACT In the fall of 1994, Applied Research