MULTICS TECHNICAL BULLETIN MTB-608
To: MTB Distribution
From: Gary M. Palter
Date: 25 January 1983
Subject: MR10.2 Multics Mail System Extensions
This MTB presents the planned extensions to the Multics mail
system to be included in the MR10.2 release. The major features
planned include:
o Integration of Mail Processing Subsystems
o Mailing Lists
o Mailing by User Name (the Mail-Table)
o Inter-Multics Mail
o Secure Forwarding with Comments in read_mail
Please direct any comments on this proposal to the author
by electronic mail to either:
Palter.Multics on either MIT or System-M
by the forum teleconferencing system to the meeting:
>udd>Multics>Palter>forums>Mail_System on System-M
or by the U.S. Postal service to:
Honeywell Information Systems, CISL
575 Technology Square
Cambridge, Mass. 02139
_________________________________________________________________
Multics Project internal working documentation. Not to be
distributed outside the project without the consent of the author
or the Director of Multics System Development.
MTB-608 MR10.2 Mail System Extensions MTB-608
INTEGRATION OF MAIL PROCESSING SUBSYSTEMS
The Extended Mail Facility PSP -- read_mail, send_mail, and
print_mail -- and Emacs RMAIL will be converted to use
mail_system_. This conversion will eliminate a large amount of
duplicated code. Additionally, as new features are added to the
mail system, these products will automatically take advantage of
them with minimal changes.
It is likely that, as part of this integration process,
several of the mail_system_ interfaces will be changed. The
Executive Mail PSP will be upgraded as necessary to insure its
continued operation.
COMMAND/REQUEST LINE ARGUMENT PROCESSING
In order to convert read_mail and send_mail, facilities for
parsing command or request line arguments must be added to
mail_system_. These facilities will provide a centralized
definition of the acceptable forms of addresses on
command/request lines.
Two facilities will be provided. The first will convert
command/request line arguments into one or more addresses. See
the info file >doc>subsystem>mail_system>addresses.gi.info for a
description of the presently accepted syntax for addresses.
The second facility will convert command/request line
arguments into one or more mailbox pathnames as required by
commands like read_mail and have_mail. For a description of the
acceptable syntax for mailbox pathnames, type:
help read_mail -section mbx_specifications
The address conversion facility in mail_system_ will accept
an extended definition for foreign addresses to allow the user to
specify an explicit routing to reach the user. The new syntax
will be:
STR {-at System1 ...} -at SystemN
specifies an address on another computer system. STR
identifies the user (or group of users) to receive the
message and is not interpreted in any way by the local
system. SystemN identifies a remote system defined in
the local system's host table; no distinction is made
between upper and lower case characters in SystemN. If
additional -at SystemJ qualifiers are present, the
message is relayed through the list of systems given
from right to left starting with SystemN; the
additional system names need not appear in the local
system's host table.
MTB-608 MR10.2 Mail System Extensions MTB-608
The -comment address qualifier will still be provided by
mail_system_ but will not be documented as it is considered
obsolete. The main purpose for supplying a comment with an
address is to supply the name of the owner of the mailbox. This
capability will be provided by a new address qualifier whose
syntax will be:
-owner STR
must appear immediately following one of the above
forms of an address and specifies the name of the owner
of the mailbox. In the header, the representation of
the owner will be:
STR <address-representation>
For example:
Gary M. Palter <gmp at MIT-MC>
Another use of the -comment qualifier is to add a comment to
a message when forwarding the message. This capability will be
provided by extending read_mail to use the forwarding with
comments facility in mail_system_ as described below.
IMPROVEMENTS TO MESSAGE EDITING IN SEND_MAIL
One of the most reported errors in send_mail occurs whenever
a user attempts to edit the header of a message using the -header
control argument of the qedx and apply requests. Any logbox or
savebox recipients in the header are converted into references to
the user's default mailbox. With the conversion to mail_system_,
the representation used in the header for the logbox and
saveboxes will be distinct from that used for the default mailbox
and this problem will no longer arise. Of course, when the
message is actually delivered to its recipients, the
representation of the logbox or savebox addresses will be
identical to that of the default mailbox to avoid recipients
attempting to reply directly to the author's logbox or savebox.
MAILING LISTS
A mailing list is a single address which causes mail to be
sent to one or more recipients. Members of a mailing list can
themselves be other mail lists.
A mailing list will be implemented as an ASCII segment with
the mls suffix. The contents of a mailing list segment are the
printed representation of addresses as they would appear in the
header of a message. For a description of the acceptable printed
representations, type:
help >doc>subsystem>mail_system>addresses.gi -section
headers
MTB-608 MR10.2 Mail System Extensions MTB-608
If more than one address is given on a single line, they must be
separated by spaces; the comma at the end of a line if more lines
follow is optional, however.
For example, the following will be a valid mailing list
segment:
Palter.Multics, Sibert.Multics
{save >udd>SiteSA>PKelley>PKelley.mlsys>outgoing},
Mail-Enthusiasts at MIT-MC
The command/request line representation of a mailing list will
be:
-mailing_list path
-mls path
specifies the pathname of a mailing list. The suffix
mls is added if necessary.
The message header representation of a mailing list will be:
{list path}
identifies a mailing list by pathname. Path is the
absolute pathname of the mailing list segment excluding
the mls suffix.
In a future release, mailing lists may be reimplemented as
inner ring segments. The extended ACL of the mailing list
segment would be used to control who may add addresses, delete
addresses, look at addresses, add/delete themselves, and send
mail to the mailing list. Commands would be provided to create
and manipulate the contents of a mailing list. In addition, a
conversion tool would be provided to allow migration from the
mailing list representation used in MR10.2.
MAILING BY USER NAME
The mail system will be upgraded to allow a user to send
mail to another user without needing to know on which project(s)
the second user is registered. In the past, this facility has
been known as the mail-table.
The preferred implementation scheme for this capability is
to use the Inquire subsystem described in MTB-585, The Multics
Inquire System: A User-Accessible, User-Maintained, Personal
User Database by Barry Margolin. However, if Inquire is not
available, a mechanism based on the present undocumented use of
the >udd>Daemon>mailboxes directory will be implemented.
MTB-608 MR10.2 Mail System Extensions MTB-608
USE OF THE INQUIRE SUBSYSTEM
The ability to send mail without needing to know a user's
project will manifest itself by a change in the definition of the
-user control argument and the addition of a new control argument
representation of an address. These new definitions will be:
-user STR
specifies a user's mail system address. STR may not
contain any whitespace characters; it may contain, at
most, one period. If STR does not contain a period, it
is interpreted as the user's Person_id and the mail
system address is obtained from that user's entry in
the Inquire database. Otherwise, it is interpreted as
Person_id.Project_id, the user's default mailbox is
used, and this control argument is equivalent to:
-mailbox >udd>Project_id>Person_id>Person_id.mbx
-last_name STR, -lnm STR
specifies the mail system address for the user with the
given last name; the search of the Inquire database is
done in a case-insensitive manner. If more than one
user has this last name, the user will be asked to
select the desired individual from the list of possible
matches.
A syntax to allow a search of the Inquire database by full
name would be desirable. However, such a capability is beyond
the scope of the current Inquire subsystem as it would involve a
linear search of the full name field which does not have a
well-defined syntax. If such a facility is ever added to
Inquire, the mail system will use it.
When sending mail, if the user's full name in the Inquire
database is marked as public, the mail system will automatically
generate a From field containing the user's full name using the
representation described above for -owner. I.e.:
From: Gary M. Palter <Palter>
Of course, if the user supplies an explicit From field, this will
not be done.
If the command interface to Inquire is available only as a
PSP, the mail system will supply two command interfaces to the
Inquire database. The first command will display a user's full
name and mail system address given their Person_id. The second
command will allow a user to change his full name and mail system
address.
MTB-608 MR10.2 Mail System Extensions MTB-608
USE OF A DIRECTORY-BASED ALTERNATIVE
If it is not possible to use Inquire in the MR10.2
time-frame, an alternative mechanism based on the present use of
>udd>Daemon>mailboxes will be implemented.
The directory >site>mail_system>mailing_addresses will be
created with directory ring brackets of (2,5). This directory
will contain one mailing list segment for each user registered on
the system. These special mailing lists will be known as mailing
addresses. Each mailing address will have two names -- the
user's Person_id exactly as it appears in the PNT and the
Person_id translated to upper-case. If two mailing addresses
would have the same all upper-case name, the first user to obtain
a mailing address would be given the all upper-case name. Any
additional names (see below) on the mailing address will always
be in all upper-case. The use of upper-case only names allows
the lookup of a user's mailing address to be done in a
case-insensitive fashion.
Two gates will be provided to access these mailing
addresses. The first gate, mailing_address_, is generally
accessible and will allow a user to change the contents of his
own mailing address (creating it if necessary) and will also
permit a user to lookup another user's mailing address. Two new
commands, set_mailing_address and display_mailing_address, will
be added to allow a user to manipulate his mailing address.
The second gate, mailing_address_admin_, is restricted to
system administrators and will allow the administrators to add
additional names to mailing addresses, create new mailing
addresses, delete existing mailing addresses, and change the
content of any user's mailing address. Additionally, the
new_user command will be upgraded to automatically initialize a
user's mailing address to their default mailbox on their default
project when adding a new user.
The ability to send mail without needing to know a user's
project will manifest itself by a change in the definition of the
-user control argument. The new definition will be:
-user STR
specifies a user's mail system address. If STR
contains exactly one period and no whitespace, it is
interpreted as Person_id.Project_id, the user's default
mailbox is used, and this control argument is
equivalent to:
-mailbox >udd>Project_id>Person_id>Person_id.mbx
Otherwise, the mailing address identified by STR is
used; lookup of the mailing address is done in a
MTB-608 MR10.2 Mail System Extensions MTB-608
case-insensitive manner. The display_mailing_address
command can be used to determine the mailing address
corresponding to a given STR.
INTER-MULTICS MAIL
As part of the ARPA TCP/IP conversion effort during late
1982, Charles Hornig implemented an inter-system mailer known as
mlsys_mailer_. This mailer allows a user to communicate with
users on other computer systems without regard to the actual
networks and protocols needed to reach those other systems. As
part of his implementation, the Extended Mail Facility and Emacs
RMAIL were upgraded to use mlsys_mailer_ when communicating with
foreign systems. However, Executive Mail was not upgraded.
In MR10.2, mlsys_mailer_ and mail_system_ will be integrated
to form a comprehensive mail system which will allow users of all
three mail PSPs to communicate with foreign users. Mail will be
transmitted between systems by one or more mailer daemon
processes. Outgoing mail will be placed into a queue which is
periodically examined by the mailer daemons. Incoming mail will
be delivered directly to the user's mailbox by the mailer daemon.
Support will also be provided to allow a Multics system to act as
a relay station between other computer systems which have no
direct path of communication.
In MR10.2, only the ARPA TCP/IP SMTP protocol and the new
inter-Multics mail protocol will be supported. The SMTP protocol
support will be provided as part of the ARPA TCP/IP PSP (or RPQ).
The inter-Multics protocol support will be provided as part of
the standard system.
The inter-Multics mail protocol will operate on either X.25
subchannels or asynchronous communcations channels; however, like
MR10.2 IMFT, reliable delivery is guaranteed only by use of X.25
subchannels. The inter-Multics protocol will preserve the AIM
access class of the mail as it is transmitted. Additionally, it
will support acknowledgements (send_mail -ack control argument)
between users on different systems.
The command/request line syntax which will be used for
foreign addresses was presented earlier in this document. The
representation of such an address in a message header will be:
STR {at System1 ...} at SystemN
The inter-system mailer requires two additional facilities
for proper operation: (1) tasking and (2) the network/host
information table.
MTB-608 MR10.2 Mail System Extensions MTB-608
The tasking facility provides for multiple execution control
points within a single process. An overview of some of the
issues pertaining to tasking on Multics may be found in MTB-596,
Tasking I, by Melanie Weaver and Charles Hornig. That MTB also
includes brief documentation of a prototype tasking facility
developed by Mr. Hornig as part of the implementation of the
ARPA TCP/IP project. This prototype facility will be installed
as an undocumented internal interface for use by the MR10.2
inter-system mailer. The facility will be installed in such a
way that a process will not incur any of the additional overhead
of tasking unless it explicitly enables tasking.
The network/host information table is a database listing all
the networks to which a Multics system is connected, the services
(eg: mail, remote login) available on those networks along with
the protocols used to implement the services, and the names of
the other systems on those networks. The network/host
information table is decribed in MTB-538, Design of a General
Interface and Implementation Structure For Use in a Networking
Environment, by Richard Kissel. A prototype table format and a
primitive set of maitenance functions were also implemented as
part of the ARPA TCP/IP project. This prototype will be used by
the MR10.2 inter-system mailer. An improved network/host
information table is presently being designed and will be the
subject of a future MTB.
SECURE FORWARDING WITH COMMENTS IN READ_MAIL
The mail_system_ interface presently provides the capability
to add comments to a message when it is forwarded. The read_mail
forward request will be upgraded to use this facility via the
addition of a new -add_comments control argument.
NEW FORWARD REQUEST CONTROL ARGUMENTS
By default, when asked to add comments, the forward request
will read the comments from the terminal. A -input_file (-if)
control argument will be provided to allow the comments to be
read from a segment. Like send_mail, the forward request will
automatically fill any comments read from the terminal but will
not fill comments read from a file. The standard -fill (-fi) and
-no_fill (-nfi) control arguments will be provided to override
this default.
If the comments are read from the terminal and are ended by
a line containing a period (.), the forward request will
automatically forward the message. If, however, the comments are
read from a file or terminal input is terminated via f (which
invokes the qedx editor) or q, the forward request will enter a
sub-request loop in which the user may examine and edit the
MTB-608 MR10.2 Mail System Extensions MTB-608
comments before actually forwarding the message. In addition,
-request_loop (-rql) and -no_request_loop (-nrql) control
arguments will be available to explicitly override the default
behavior.
FORWARD REQUEST SUB-REQUEST LOOP
As mentioned above, a sub-request loop will be implemented
within the forward request to allow the user to edit a comment
before transmission. The major requests within this sub-request
loop will be:
print, pr, p
prints the comment text.
print_original, pro
prints the message(s) which are being forwarded.
qedx, qx
edits the comment text using the Multics qedx editor.
apply, ap
permits editing of the comment text using any Multics
editor.
send
forwards the message(s) and exits the forward request's
sub-request loop.
quit, q
exits the forward request's sub-request loop without
forwarding the message(s).
Unlike the send request in send_mail, the send sub-request
listed above terminates the forward request in addition to
actually forwarding the message. This operation was chosen
because it is not possible to change the list of recipients of
the forwarded message and, therefore, it does not make sense to
ask that the message be transmitted more than once.
SECURING THE FORWARDING OPERATION
When a message is forwarded, the mail system adds several
header fields to indicate that this has taken place. In
addition, if comments are added, these comments are added to the
message header. However, as presently implemented, forwarding is
not a secure operation. In other words, it is possible to
construct a message which appears to have been forwarded by
someone else.
MTB-608 MR10.2 Mail System Extensions MTB-608
In order to secure the forwarding operation, the entire
mail_system_ interface must be moved to a lower ring. Then,
mail_system_ will be able to verify that it actually sent a
message and can, therefore, trust the forwarding information
contained therein. For messages sent before the securing of the
interface or messages sent from an outer ring, mail_system_ will
not trust the forwarding information. In these cases,
mail_system_ will indicate to the user that all forwarding
operations were actually performed by the last person to transmit
the message.
As part of the movement of mail_system_ to a lower ring
(ring 2), the format of messages stored in the mailbox will be
changed to a binary representation. This change will greatly
improve the performance of mail_system_ when reading messages
from a mailbox. As part of this conversion, the old mail command
will be converted to use the mail_system_ interface to insure
that it will be able to continue to read the messages stored in a
mailbox. Additionally, the Executive Mail PSP will have to be
modified to treat all data returned by mail_system_ as read-only
as said data will actually reside in the lower ring.
OTHER IMPROVEMENTS
In addition to the major features described above, bugs will
be fixed and minor suggestions will be implemented as an integral
part of the project. The exact list of fixed bugs and
implemented suggestions will be included with the appropriate
MCRs.
Several other enhancements will be made to the mail system
if time permits. These enhancements include (in no particular
order):
o Acknowledgements of forwarded messages:
The -acknowledge (-ack) control argument will be added
to read_mail's forward request to cause an
acknowledgement to be sent to the user who forwarded
the message when it is read by each of the recipients
of the forwarding.
o The blind carbon copies (bcc) field:
The bcc field will be supported in send_mail through
the use of the new -bcc control argument and the new
bcc request. Recipients in the bcc field receive a
copy of the message which indicates that they were
blind recipients of the message. The primary and
secondary recipients of the message are not informed
that there are any blind recipients of the message.
MTB-608 MR10.2 Mail System Extensions MTB-608
This capability is already present within mail_system_
but additional work would be necessary to make it
available from send_mail.
o Secure annotation of messages:
The user will be permitted to edit the header and text
of a message already present in a mailbox and then
rewrite that message into the mailbox. The fact that
the message has been altered is recorded as part of the
updated message in an unforgeable manner. This feature
requires the enhancements to the ring-1 message segment
primitives described in Ned Kittlitz's draft MTB --
>udd>Multics>Kittlitz>mseg>message_segment.mtb on
System-M.
o Named groups of messages in read_mail:
A new request, group, will be added to allow the
definition of a named group of messages. This group
can then be used in later requests through the new
-group control argument. For example:
group old_mail -before 10/1/81 -from Palter
-subject /mail/
log -group old_mail -delete
o Sending mail to a forum meeting:
A new type of address will be added to the mail system
which specifies that a message should be delivered to a
specific forum meeting. This address will be specified
on command/request lines as:
-meeting path (-mtg path)
It will appear in a message header as:
{forum path}
o Unseen messages in read_mail:
The mail system will be extended to record an unseen
flag for each message. This flag will be reset by the
read_mail requests print, reply, write, save, forward,
append, and preface and by Executive Mail and Emacs
RMAIL when they display a message on the terminal. New
message specifiers will be added to read_mail to allow
the selection of all unseen messages (all_unseen,
unseen, au), the first unseen message (first_unseen,
fu), the next unseen message (next_unseen, nu), etc.
This capability is a generalization of the new
transaction specifier in forum. Indeed, present forum
development plans call for the inclusion of transaction
specifiers similar to the message specifiers described
here when the forum meeting format is upgraded to allow
the bitmap of seen transactions.
MTB-608 MR10.2 Mail System Extensions MTB-608
o New send_mail requests:
Three new requests -- include_original,
include_authors, and include_recipients -- will be
added to send_mail. These requests, available only
within the send_mail created by a reply request, allow
a user to insert the original message text into the
reply or to add the authors or recipients of the
original message as recipients to the reply after
reaching the send_mail request loop. These requests
are analagous to the -include_original,
-include_authors, and -include_recipients control
arguments of the reply request. The send_mail requests
allow a user to still perform one of these operations
even when they forget to use the appropriate reply
control arguments.
o Conversion of send_mail to use qedx_:
The qedx request in send_mail will be changed to use
the new qedx_ subroutine interface rather than its own
internal qedx-like editor. This new version of the
qedx request will require the use of the write (w)
request to reflect changes made to the message back to
send_mail. In addition, the quit (q) request will
query if there are modified buffers and a new
quit-force (:q, Q) request will be added which does not
query. Finally, the read (r) and write (w) requests
will be changed to not accept pathnames and the
read-insert (:r) and write-copy (:w) requests will be
added to achieve the old behavior of read and write
with pathnames. The send_mail qedx request and the
qedx command presently have a major incompatibility
with respect to the behavior of the quit request. As a
result, users who frequently use send_mail qedx often
forget to save their buffers when using the qedx
command and often lose large amounts of work.
Requiring the write request to save changes in
send_mail qedx is, however, a severely imcompatible
change which can not be made without providing the user
with some protection; thus, the quit request will query
when there are modified buffers.
MTB-608 MR10.2 Mail System Extensions MTB-608
IMPLEMENTATION SCHEDULE
The following table presents a rough estimate of the time
required to implement each of the enhancements described above:
Feature Time
Conversion of Extended Mail to 8 person-weeks
mail_system_
Conversion of Emacs RMAIL to 4 person-weeks
mail_system_
Upgrading Executive Mail 4 person-weeks
Mailing lists 2 person-weeks
Mailing by User Name (Inquire) 1 person-week
Inter-Multics Mail 6 person-weeks
Secure Forwarding with Comments 4 person-weeks
in read_mail
Total 29 person-weeks
Other Improvements
Acknowledgements of Forwarded 1 person-day
Messages
The Blind Carbon Copies Field 2 person-days
Secure Annotation of Messages 1 person-week
Named Groups of Messages in 1 person-week
read_mail
Sending Mail to a Forum 2 person-days
Meeting
Unseen Messages in read_mail 1 person-week
New send_mail requests 1 person-week
Conversion of send_mail to use 1 person-week
qedx_
Total for Other Improvements 6 person-weeks
Total including Other Improvements 35 person-weeks
If the >site>mail_system>mailing_addresses mechanism for
mailing by user name is to be used instead of Inquire, an
additional person-week should be added to the above total
estimates.
The conversion of Emacs RMAIL to use mail_system_ will be
performed by Barry Margolin in parallel with my conversion of the
Extended Mail Facility. The upgrading of Executive Mail should
be undertaken by Dave Schimke who is the present maintainer of
that product and should also be done in parallel with my
conversion of the Extended Mail Facility. This parallel effort
should reduce the elapsed time for these three tasks from sixteen
to eight weeks.
MTB-608 MR10.2 Mail System Extensions MTB-608
If there is not enough time to implement all of the
enhancements listed above, those items listed as Other
Improvements will be omitted as necessary.