Extended Objects MULTICS TECHNICAL BULLETIN MTB-615
Date: 6 March 1983
From: Jay Pattin
To: MTB Distribution
Subject: A mechanism for managing extended objects
ABSTRACT
This MTB describes a mechanism for manipulating "Extended
Objects" in the file system. The mechanism provides fully
general support for common operations, and can be extended to
handle arbitrary user-defined objects without any system changes.
Comments on this MTB should be directed to the author
by Multics mail to:
Pattin.PDO on MIT
Pattin.Multics on System-M
or to the Forum meeting:
>udd>Multics>Pattin>meetings>Extended_Objects on
System-M
________________________________________
Multics Project internal working documentation. Not to be
reproduced or distributed outside the Multics project.
Extended Objects MTB-615
INTRODUCTION
A few examples of the extended objects which require such
manipulations are mailboxes, message segments, forum meetings,
and the MSFs which will be used to implement databases in the
first Data Management implementation. Today, these are handled
in ad-hoc and incompatible ways: special commands and
subroutines, and special-case code in standard system commands
and subroutines.
The reason for these special cases is twofold: (1) most
extended objects exist in an inner ring, where normal hcs_ calls
from the user ring cannot touch them, and (2) some validation is
usually necessary before the operation can be performed.
Additionally, when an extended object uses the extended ACL
mechanism, the standard ACL commands cannot currently be used.
THE MECHANISM
The basic mechanism depends on the extended objects having a
specific suffix which identifies their object type. All such
objects in the standard system have defined suffixes today:
.mbx, .ms, .forum, etc. For each type of object, there is a
program called suffix_XXX_, where XXX is the suffix of the
object. The entrypoints in suffix_XXX_ mimic the corresponding
hcs_ entries. Additionally, suffix_XXX_ contains an entrypoint
to verify that a given storage system object is of that type, and
also an entry that returns information about the object type.
The file system commands will be changed to call a
subroutine called object_type_ where they now call hcs_. The
object_type_ module will look for an owning suffix_XXX_ routine
and call it. If no suffix_XXX_ can be found, or if the
suffix_XXX_ entrypoint rejected the object, the hcs_ entry will
be called.
Unfortunately, Data Management Files are not distinguishable
by having a known suffix. However, the implementation of the
suffix mechanism has only one module, object_type_, which
determines the type of an object based on its suffix. It is
quite simple to modify this module to determine whether or not an
object is a DM file if no suffix_XXX_ module can be found for an
object.
The suffix_XXX_ subroutine for an object should have
entrypoints for all the operations is can support. The planned
operations are validate, copy, delete, chname, (get set)_switch,
(get set)_max_length, set_bit_count, and ACL manipulation. In
Extended Objects MTB-615
addition, suffix_info and list_switches entrypoints will be
provided to return information about the object type.
Name Changing
Name changing is simple. The standard rename command and
the copy_names_ and nd_handler_ subroutines will be changed to
call object_type_$chname_file instead of hcs_$chname_file. There
should be no need to modify other callers of hcs_$chname_file.
Deleting Objects
For deletion, modifying the delete_$path subroutine to call
object_type_$delentry_file instead of hcs_$delentry_file should
be sufficient. The dl_handler_ program will need to call
object_type_ to retrieve and set the values of the safety and
copy switches. All extended objects will be treated as segments,
thus delete_dir will not delete a forum meeting even though it is
a directory, but delete will.
Branch Attributes
Changing the rest of the branch attibutes, such as switches
(safety, copy, damaged, etc.), the max_length, and the bit count
is also simple. The copy/move, set_bit_count, switch_on, and
switch_off commands and the copy_seg_ subroutine can be easily
changed to call object_type_ instead of hcs_. There seems to be
no need to change other programs which currently set these
attributes. Also, it should not be necessary to update the
obsolete switch commands which were replaced by the swn/swf
commands.
Because extended objects may wish to define their own
switches which should be accessible by using the
switch_on/switch_off commands, those commands will be changed to
set switches by calling object_type_$set_switch. This routine
will return error_table_$argerr if the named switch is not
supported for the object. The commands will be changed to detect
that code and give a better error message. However, the same
restriction about using starnames with the set_acl command (See
below) may want to be enforced when a non-standard switchname is
used.
Copying And Moving Objects
Copying, and by implication moving, is difficult.
Basically, the copy_seg_ subroutine will be changed to first
check for name duplication on the target and call nd_handler_ if
appropriate. Thus, suffix_XXX_ entries never have to call
nd_handler_. Then it will call object_type_$copy. The guts of
the current copy_seg_, the part that creates the target and
Extended Objects MTB-615
copies the contents, will be accessible as copy_seg_$copy_seg_raw
and will be called by object_type_ if no suffix_XXX_ is found.
Copying names and acls is already taken care of by the
modifications to copy_names_ and copy_acl_. Changing the copy
and move commands to change branch attributes using object_type_
instead of hcs_ completes the effort. The copy_dir_ subroutine
should be changed to refuse to move directories that are really
extended objects - DM Files for example.
Because a user may wish to force the command to use the
standard hcs_ entries (See "Avoiding object_type_" below), a new
entrypoint into copy_seg_ is needed which will not call
object_type_. In order to stop the proliferation of entrypoints
in copy_seg_ which differ in meaning by the values of switches, a
new copy_ entry into copy_seg_ will be written which takes a set
of flags (much like delete_).
ACL Manipulation
ACL manipulation is also a touchy area. The proposed
solution is predicated on two conditions: 1) objects either use
ACLs, or they use extended ACLs - it is never meaningful for the
user to specify both; and 2) DM files do not use extended ACL's.
These conditions allow us to modify the existing ACL commands
(set_acl, delete_acl, and list_acl) to cope with extended ACLs.
In all of these commands, the way directories and MSFs which are
not extended objects are handled will not be changed.
The delete_acl command need only be modified to use
object_type_$delete_acl_entries instead of
hcs_$delete_acl_entries. The list_acl command can be modified to
use object_type_$list_acl, and to show either ACLs or extended
ACLs as appropriate (as determined by a call to
object_type_$suffix_info).
The set_acl command is tricky. While it is simple to
determine whether or not an object is an extended object, the use
of starnames can obscure the user's intent. However, since all
objects which use extended ACLs have suffixes (This is the reason
for restricting DM files to the standard rew terms.), the
following rules can be developed.
If a starname is given, and its last component contains a
star convention character ('*' or '?'), then assume regular
ACLs are intended and do not attempt to change the ACL for
extended objects. If the last component contains no star
characters, then use the results of object_type_$suffix_info
to determine how to proceed. If a suffix_XXX_ is found for
a starname, then only operate on objects owned by that
module.
Extended Objects MTB-615
I believe that this rule is not too difficult to document and
have people understand, and the fact that "sa Pattin.mbx" will
work is certainly worth a little explaining.
Both the list_acl command and the set_acl command will use
the segment_acl structure in calls to object_type_. If the
object uses extended ACLs, the set_acl command will not touch the
word of standard modes, this is the responsibility of the
suffix_XXX_ routine. The list_acl command will never look at the
standard modes - if -force_no_type is specified, hcs_ is called
instead.
In order to make the command interface a little bit more
understandable, a -type control argument can be added to specify
what type of objects are to be affected. The command line:
sa ** -type mbx aow *
would be exactly equivalent to the command line:
sa **.mbx aow *
An exception would occur when the type is dm_file (dmf), because
these are not identifiable by virtue of a suffix. Specifying one
of the branch-type control arguments (-dr, -sm, -msf, or -dmf)
when using extended objects, regardless of whether or not they
use extended acls, is an error. These arguments may be used with
extended objects when the -force_no_type control argument is
given.
The list and status commands
As a further modification, the status command can be changed
to obtain information through object_type_. Currently, that
information is the type name, the effective mode, max length, and
switch values. Also, the list command can be changed to group
objects by types (headings for "Mailboxes" and "Forum Meetings"
as well as "Segments" and "Directories"), and to show
type-specific data, such as extended ACLs. This should not be
the default behavior as there is considerable expense involved.
The status command should be modified to report on
information specific to an object type. For instance,
records-used for a mailbox could be the number of messages in the
mailbox. This requires considerable design and implementation
work and is therefore being left as the subject for a later MTB.
It will probably be merged in with Benson Margulies' fs_info_
subroutine.
Miscellaneous
Modifications to some utility commands and active functions
are needed. Two new keywords to the "exists" command will be
added:
Extended Objects MTB-615
exists type FOO
will return true iff a suffix_FOO_$validate can be found, and
exists object FOO STARNAME
will return true iff a suffix_FOO_$validate can be found and
there is an object matching STARNAME that is of type FOO.
Presumbly, the Data Management group will add new keywords for DM
files.
A -type TYPE control argument will be added to the entries
and files commands to return the names of files of that type in a
directory. TYPE is used as a suffix, so "files ** -type foo" is
exactly equivalent to "files **.foo -type foo".
The suffix_XXX_ Subroutine
The suffix_XXX_ subroutine for a particular object may be a
gate, or it may be a user ring program which calls the
appropriate gates. It must not have additional entrypoints not
used by the standard mechanism, since additional standard
entrypoints may be defined in the future; indeed, it may need to
be a standalone program in order to ensure that other entrypoints
in its bound segment do not conflict either. If an entrypoint
(such as chname_file) is not found in suffix_XXX_, the calling
program (normally object_type_) will try using the normal hcs_
entrypoint instead.
It is the responsibility of suffix_XXX_ to ensure that the
object it is told to operate on is, in fact, the right type of
object (rather than just a random segment named foo.mbx). If
suffix_XXX_ returns error_table_$not_seg_type, the normal hcs_
operation should be tried instead.
Initially, each suffix_XXX_ subroutine should contain the
following entrypoints:
add_acl_entries list_acl
chname_file list_switches
copy replace_acl
delentry_file set_bit_count
delete_acl_entries set_switch
get_max_length set_max_length
get_switch suffix_info
get_user_access_modes validate
All of the entries except copy, (set get)_switch,
list_switches, suffix_info, and validate are indentical to the
corresponding hcs_ entry. The copy entrypoint takes the same
Extended Objects MTB-615
arguments as copy_seg_ except for the caller_name, since
copy_seg_ itself will be calling nd_handler_. Detailed
descriptions of the other entrypoints are provided in the
Appendix. Basically, the set and get_switch entrypoints take a
switch name and either return a value or change the value as
appropriate. The validate entrypoint takes a directory name and
an entryname and returns an error code of 0 if the object is of
the correct type, or error_table_$not_seg_type if it is not. The
suffix_info entrypoint returns various generic information about
the object type, such as its name (singular and plural) and
whether it uses regular ACLs or extended ACLs. The list_switches
entry is used to return the names of switches supported by the
object type.
THE OBJECT_TYPE_ SUBROUTINE
A mechanism which makes it easier to write programs using
the suffix_XXX_ entrypoints is this: an object_type_ subroutine,
which has entrypoints corresponding to all the defined
suffix_XXX_ entrypoints, and would try to find, and call, the
suffix_XXX_ entry first, and, failing that, would call the
appropriate file_manager_ entrypoint if the object is a DM file,
and would otherwise call the hcs_ entry instead. The algorithm
used inside object_type_ to determine what entry to call is as
follows:
1) If the entry name does not contain a ".", go to step 5.
2) Call hcs_$make_entry looking for suffix_XXX_$<operation>.
If this call fails, go to step 5.
3) Call the entry constructed in step 2.
4) If the error code returned in step 3 is not
error_table_$not_seg_type, we are finished. Return to
caller.
5) Now, determine if the entry is a DM file. Calling
file_manager_ is expensive, so do cheap tests first. Call
hcs_$get_dir_ring_brackets and hcs_$status_minf. If the
entry is not an MSF, or it if does not have the proper ring
brackets for DM files, go to step 7.
6) Call the appropriate file manager entry point. If the
return code is not error_table_$not_seg_type, return to
caller.
7) Call the appropriate hcs_ entry and return to caller.
Extended Objects MTB-615
Note that determining the type of an object is not trivial.
Entrynames with suffixes require at least one call to
hcs_$make_entry. If no suffix_XXX_ can be found, then
determining whether or not the object is a DM file is also
expensive. In addition to making it a little simpler to write
delete, rename, and so forth, object_type_ would also have
another advantage: it could be hard-coded to know about certain
suffixes, and call things like suffix_mbx_ (which it KNOWS to be
in the system) through links, rather than calling
hcs_$make_entry, making it a little faster.
The object_type_$suffix_info and object_type_$list_switches
entrypoints are slightly different from their counterparts in the
suffix_XXX_ modules. The suffix_info entry takes a directory,
entryname, and a code so that it can 1) figure out which
suffix_XXX_ to call, and 2) handle the case where the object is
not owned by a suffix_XXX_. The list_switches entrypoint takes
an entryname and a code for similar reasons. It does not take an
entryname because it always returns an error if no suffix_XXX_ is
found. As an additional utility, an object_type_$make_entry
subroutine is provided to create an entry variable given an
entryname and an entrypoint name ("list_acl" for example).
AVOIDING OBJECT_TYPE_
There are times when calling object_type_ is probably not
the right thing to do. For instance, when copying a damaged
mailbox from ring-1 comamnd level. For this purpose, all of the
modified file system commands will take a -force_no_type (-fcnt)
control argument which will cause the commands to call hcs_
instead of object_type_. This can be used with the list_acl
command to display the real ACL on an object, which would
otherwise be un-obtainable. As a gratuitous modification, "la
-force_no_type" will also list the ACL on MSF directories rather
than calling msf_manager_.
The alert reader may have noticed that not all commands get
to decide whether or not hcs_ of object_type_ gets called. Thus,
the copy_, delete_, and nd_handler_ routines must be told to not
call object_type_. copy_, copy_dir_, delete_, and
nd_handler_$switches (MCR pending) will take a switch which says
whether or not to call hcs_.
BENEFITS
The three major benefits of this mechanism are: (1)
centralization of the code to handle extended objects, (2)
Extended Objects MTB-615
elimination (or at least desupport) of a wide set of existing
tools which behave inconsistently, and (3) easy extension to
arbitrary user applications.
The suffix_XXX_ mechanism makes it very easy to implement
trivial secure applications using rings. For instance, it would
be quite simple to implement acs segments as inner ring objects,
and this would eliminate the problems we have today which could
result from someone writing data into an acs segment if the max
length was accidentally not set.
By eliminating the special purpose commands (mbrn, mbsa,
fapt, and so forth), a large body of code is eliminated, and the
interfaces are made consistent for all types of objects. No more
complaints about how mbsa ought to implement -replace, and so
forth.
APPLICATIONS
There are numerous applications for this mechanism. Some
which come to mind are listed below; in addition, users have
created this sort of thing in the past (the Multics/Cray
connection has some examples, Olin thinks) and will want to
again.
mailboxes, message segments
Presently, we have a plethora of silly commands for
these extended ACLs. They could be readily made
obsolete with this mechanism, with only a minimum of
work (on top of implementing the commands themselves,
which are desirable for other reasons) required to
implement the appropriate suffix_mbx_ and suffix_ms_
subroutines.
forum meetings
Today, we have forum_add_participant,
forum_add_project, forum_make_public, and so forth.
They behave differently from normal ACL commands, for
no particular reason save that this was how they were
implemented. We also have no way to rename meetings,
and we cannot delete meetings except with forum_delete.
All this would be solved by using extended ACLs and
suffix_XXX_.
logical volume access control segments
A lot of effort has gone into figuring out some way of
mapping the three available letters ("R", "E", and "W")
into sensible modes ("executive" ??) for logical
volumes. It would seem much more sensible and
Extended Objects MTB-615
extensible to use extended ACLs, which put the entire
character set at your disposal.
RCP (and others) acs segments
The same argument applies as for logical volume
segments. Additionally, thr multi-ring RCP environment
would benefit considerably from not being misled by the
ways the normal rew modes change in relation to the
segment's ring brackets and your current validation
level. Additionally, if acs segments were managed in
ring one, it would remove the present ability for a
user to write into one which didn't have its max length
set properly; indeed, it would make it straightforward
to insure that the max length was always set properly.
WORK REQUIRED
Most of the work for this has been done. The following
modules have been modified (or written): acl_commands_
(replacement for set_acl), rename, delete, delete_, copy,
copy_acl_, copy_names_, copy_seg_, dl_handler_, entries, exists,
nd_handler_, set_bit_count, set_max_length, switch_on,
object_type_, status, suffix_mbx, suffix_ms_, and suffix_forum_.
The list and copy_dir (copy_dir_) commands have not been touched.
The interface to the file_manager_ for DM Files is being
written by Steve Herbst. The necessary inner_ring support for
the mailbox and message segment primitives is being written by
Gary Palter.
Extended Objects MTB-615
SUBROUTINE DESCRIPTIONS
NAME: copy_
Function: Copies storage system objects. This subroutine
replaces the copy_seg_ subroutine.
Usage:
dcl copy_ entry (char (*), char (*), char (*), char (*),
char (*), bit (36) aligned, bit (1) aligned, fixed bin
(35));
call copy_ (source_dir, source_name, target_dir,
target_name, caller_name, switches, error_switch,
code);
where
source_dir (Input)
is the name of the directory containing the entry to be
copied.
source_name (Input)
is the name of the entry to be copies.
target_dir (Input)
is the name of the directory in which the copy is to be
created.
target_name (Input)
is the entry name of the copy.
caller_name (Input)
is the name of the caller of copy_. This string is used in
calls to nd_handler_.
switches (Input)
is a string of switches described by the following
structure, which is declared in copy_options.incl.pl1. This
should be passed as an argument by using the PL/1 string
builtin (string (copy_options)).
dcl 1 copy_options aligned,
2 brief bit (1) unaligned,
2 no_name_dup bit (1) unaligned,
2 raw bit (1) unaligned,
2 mbz bit (33) unaligned;
Extended Objects MTB-615
where
brief
is turned on to prevent messages of the form "Bit count
inconsistent with current length." from being printed.
no_name_dup
is turned on to prevent nd_handler_ from being called
when the target segment already exists.
raw
is turned on to prevent the object_type_ subroutine from
being used. If this switch is on, all calls wil be made
through hcs_.
error_switch (Output)
In the event of an error, this switch is set to "0"b if the
error occurred on the source, and to "1"b if the error
occurred on the target.
code
is a standard storage system error code.
______________________________
Name: suffix_XXX_$get_switch
Function: Returns the value of a storage system switch for an
entry.
Usage:
dcl suffix_XXX_$get_switch entry (char (*), char (*), char
(*), bit (1) aligned, fixed bin (35));
call suffix_XXX_$get_switch (dirname, entryname,
switch_name, value, code);
where
dirname (Input)
is the pathname of the containing directory.
entryname (Input)
is the name of the entry.
switch_name (Input)
is the name of the switch whose value is desired. This may
Extended Objects MTB-615
be one of "copy", "complete_volume_dump", "damaged",
"incremental_volume_dump", "safety", "synchronized", or any
switch defined by the extended object type.
value (Output)
is the value of the requested switch.
code (Output)
is a standard error code. It should be set to
error_table_$argerr if switch_name is invalid.
______________________________
Name: suffix_XXX_$set_switch
Function: Sets the value of a storage system switch for an
entry.
Usage:
dcl suffix_XXX_$set_switch entry (char (*), char (*), char
(*), bit (1) aligned, fixed bin (35));
call suffix_XXX_$set_switch (dirname, entryname,
switch_name, value, code);
where
dirname (Input)
is the pathname of the containing directory.
entryname (Input)
is the name of the entry.
switch_name (Input)
is the name of the switch whose value is to be set. This
may be one of "copy", "complete_volume_dump", "damaged",
"incremental_volume_dump", "safety", "synchronized", or any
switch defined by the extended object type.
value (Input)
is the value the switch is to be set to.
code (Output)
is a standard error code. It should be set to
error_table_$argerr if switch_name is invalid.
Extended Objects MTB-615
______________________________
Name: suffix_XXX_$suffix_info
Function: Returns information about an extended object type.
Usage:
dcl suffix_XXX_$suffix_info entry (ptr);
call suffix_XXX_$suffix_info (suffix_info_ptr);
where
suffix_info_ptr (Input)
is a pointer to the following info structure, defined in the
include file suffix_info.incl.pl1.
dcl 1 suffix_info aligned,
2 version char (8),
2 type_name char (32),
2 plural_name char (32),
2 flags unaligned,
3 standard_object bit (1) unaligned,
3 extended_acl bit (1) unaligned,
3 has_switches bit (1) unaligned,
3 mbz1 bit (33) unaligned,
2 modes char (36),
2 max_mode_len fixed bin;
where
version
is the version of this structure. This should be set to
SUFFIX_INFO_VERSION_1, which is also declared in the include
file.
type_name
is the singular name of the object type (e.g. "mailbox").
plural_name
is the plural name of the object type (e.g. "mailboxes").
standard_object
is set by object_type_ when no suffix_XXX_ can be found for
the entryname.
extended_acl
is a switch telling whether or not the object type uses
Extended Objects MTB-615
standard or extended ACLs. The switch should be on for
extended ACLs, and off otherwise.
has_switches
is on if the object type supports the (get set)_switch
entries.
modes
is a string containing the extended modes for the object
type. This string contains one character for each extended
mode bit. The position of the character in the string
indicates which bit in the ACL represents that mode.
max_mode_len
is the maximum number of modes on a single object of this
type. This is used by the list_acl command for formatting.
Notes: When object_type_$suffix_info is called and no
suffix_XXX_$suffix_info can be found, the structure is filled in
by object_type_ as follows: standard object is set to "1"b,
extended_acl and has_switches are set to "0"b, and max_mode_len
to 3. If entryname is not a starname, type_name and plural_name
are set to whichever of "Segment(s)", "Directory(ies)", or
"Multi-Segment File(s)" is appropriate (links are chased), and
modes is set to "rew" or "sma" as appropriate.
______________________________
Name: suffix_XXX_$list_switches
Function: Returns a list of switches supported by the object
type.
Usage:
dcl suffix_XXX_$list_switches entry (ptr, ptr);
call suffix_XXX_$list_switches (area_ptr, switch_list_ptr);
where
area_ptr (Input)
is a pointer to an area where the return structure may be
allocated.
Extended Objects MTB-615
switch_list_ptr (Output)
is a pointer to the switch_list structure, declared in
suffix_info.incl.pl1 and described below.
dcl 1 switch_list aligned,
2 version char (8),
2 switch_count fixed bin,
2 switch_name_count fixed bin,
2 switches (switch_list.switch_count),
3 name_index fixed bin,
3 name_count fixed bin,
3 default_value bit (1) aligned,
3 mbz1 bit (36) aligned,
2 names (switch_list.switch_name_count)
char (32);
where
version
is the version of this structure. This should be set to
SWITCH_LIST_VERSION_1, which is also declared in the include
file.
switch_count
is the number of switches defined for this object type.
switch_name_count
is the total number of names of the switches - a switch can
have multiple names.
name_index
is the index into the switch_list.names array of the first
name for this switch.
name_count
is the number of names for this switch. The names for this
switch are located in switch_list.names (name_index) through
switch_list.names (name_index + name_count - 1).
default_value
is the default setting for this switch when the object is
created.
names
is the array of switch names.
______________________________
Name: object_type_$make_entry
Extended Objects MTB-615
Function: constructs an entry variable to a specified
suffix_XXX_ subroutine entry for a specified object.
Usage:
dcl object_type_$make_entry entry (char (*), char (*),
entry, fixed bin (35));
call object_type_$make_entry (entryname, operation,
entry_to_call, code);
where
entryname (Input)
is the entryname of the object for which the suffix_XXX_
entry is to be made.
operation (Input)
is the entrypoint name of the entry to be constructed.
entry_to_call (Output)
is the entry variable constructed.
code (Output)
is a standard error code.
Notes: If code is non-zero, then the value of entry_to_call is
not set. Otherwise, entry_to_call contains an entry value for
suffix_XXX_$operation, where XXX is the suffix of entryname.
______________________________
Name: object_type_$validate
Function: Verifies that a storage system object is of the
correct type and does not merely have a name of the correct type.
Usage:
dcl object_type_$validate entry (char (*), char (*), fixed
bin (35));
call object_type_$validate (dirname, entryname, code);
where
Extended Objects MTB-615
dirname (Input)
is the pathname of the containing directory.
entryname (Input)
is the entryname of the object whose type is in question.
code (Output)
is a standard error code. It will either be 0 if the object
is of the correct type, or error_table_$not_seg_type if it
is not of the correct type or if no suffix_XXX_ can be
found.