MULTICS TECHNICAL BULLETIN                                MTB-608

To:       MTB Distribution

From:     Gary M.  Palter

Date:     25 January 1983

Subject:  MR10.2 Multics Mail System Extensions

     This MTB presents the planned extensions to the Multics mail
system to be included in  the MR10.2 release.  The major features
planned include:

  o  Integration of Mail Processing Subsystems

  o  Mailing Lists

  o  Mailing by User Name (the Mail-Table)

  o  Inter-Multics Mail

  o  Secure Forwarding with Comments in read_mail

Please direct any comments on this proposal to the author

     by electronic mail to either:

          Palter.Multics on either MIT or System-M

     by the forum teleconferencing system to the meeting:

          >udd>Multics>Palter>forums>Mail_System on System-M

     or by the U.S.  Postal service to:

          Honeywell Information Systems, CISL
          575 Technology Square
          Cambridge, Mass.  02139


Multics  Project  internal  working  documentation.   Not  to  be
distributed outside the project without the consent of the author
or the Director of Multics System Development.

MTB-608           MR10.2 Mail System Extensions           MTB-608


     The Extended Mail Facility  PSP -- read_mail, send_mail, and
print_mail  --   and  Emacs  RMAIL  will   be  converted  to  use
mail_system_.  This  conversion will eliminate a  large amount of
duplicated code.  Additionally, as new  features are added to the
mail system, these products  will automatically take advantage of
them with minimal changes.

     It  is  likely that,  as part  of this  integration process,
several  of  the mail_system_  interfaces  will be  changed.  The
Executive Mail  PSP will be  upgraded as necessary  to insure its
continued operation.


     In order to convert  read_mail and send_mail, facilities for
parsing  command  or  request  line arguments  must  be  added to
mail_system_.   These  facilities   will  provide  a  centralized
definition   of   the   acceptable    forms   of   addresses   on
command/request lines.

     Two  facilities will  be provided.   The first  will convert
command/request line  arguments into one or  more addresses.  See
the info file  >doc>subsystem>mail_system> for a
description of the presently accepted syntax for addresses.

     The  second  facility   will  convert  command/request  line
arguments  into  one or  more  mailbox pathnames  as  required by
commands like read_mail and have_mail.   For a description of the
acceptable syntax for mailbox pathnames, type:
          help read_mail -section mbx_specifications

     The address conversion facility  in mail_system_ will accept
an extended definition for foreign addresses to allow the user to
specify an  explicit routing to  reach the user.   The new syntax
will be:

     STR {-at System1 ...} -at SystemN
          specifies an  address on another  computer system.  STR
          identifies the user (or group  of users) to receive the
          message and is not interpreted  in any way by the local
          system.  SystemN identifies a  remote system defined in
          the local  system's host table; no  distinction is made
          between upper and lower case characters in SystemN.  If
          additional  -at SystemJ  qualifiers  are  present,  the
          message  is relayed  through the list  of systems given
          from   right  to   left  starting   with  SystemN;  the
          additional  system names  need not appear  in the local
          system's host table.

MTB-608           MR10.2 Mail System Extensions           MTB-608

     The  -comment address  qualifier will  still be  provided by
mail_system_  but  will not  be  documented as  it  is considered
obsolete.   The  main purpose  for  supplying a  comment  with an
address is to supply the name  of the owner of the mailbox.  This
capability  will  be provided  by a  new address  qualifier whose
syntax will be:

     -owner STR
          must  appear  immediately  following one  of  the above
          forms of an address and specifies the name of the owner
          of the  mailbox.  In the header,  the representation of
          the owner will be:
               STR <address-representation>
          For example:
               Gary M. Palter <gmp at MIT-MC>

     Another use of the -comment qualifier is to add a comment to
a message  when forwarding the message.   This capability will be
provided  by  extending  read_mail  to  use  the  forwarding with
comments facility in mail_system_ as described below.


     One of the most reported errors in send_mail occurs whenever
a user attempts to edit the header of a message using the -header
control argument of  the qedx and apply requests.   Any logbox or
savebox recipients in the header are converted into references to
the user's default mailbox.  With the conversion to mail_system_,
the  representation  used  in  the  header  for  the  logbox  and
saveboxes will be distinct from that used for the default mailbox
and  this  problem will  no  longer arise.   Of course,  when the
message   is   actually   delivered   to   its   recipients,  the
representation  of  the  logbox  or  savebox  addresses  will  be
identical  to  that of  the default  mailbox to  avoid recipients
attempting to reply directly to the author's logbox or savebox.


     A mailing list  is a single address which  causes mail to be
sent to  one or more  recipients.  Members of a  mailing list can
themselves be other mail lists.

     A mailing list will be  implemented as an ASCII segment with
the mls suffix.   The contents of a mailing  list segment are the
printed representation  of addresses as they  would appear in the
header of a message.  For a description of the acceptable printed
representations, type:
          help  >doc>subsystem>mail_system>  -section

MTB-608           MR10.2 Mail System Extensions           MTB-608

If more than one address is given  on a single line, they must be
separated by spaces; the comma at the end of a line if more lines
follow is optional, however.

     For  example,  the following  will be  a valid  mailing list

          Palter.Multics, Sibert.Multics
          {save >udd>SiteSA>PKelley>PKelley.mlsys>outgoing},
          Mail-Enthusiasts at MIT-MC

The  command/request line  representation of a  mailing list will

     -mailing_list path
     -mls path
          specifies the  pathname of a mailing  list.  The suffix
          mls is added if necessary.

The message header representation of a mailing list will be:

     {list path}
          identifies  a mailing  list by  pathname.  Path  is the
          absolute pathname of the mailing list segment excluding
          the mls suffix.

     In a  future release, mailing lists  may be reimplemented as
inner  ring  segments.   The  extended ACL  of  the  mailing list
segment would  be used to  control who may  add addresses, delete
addresses,  look  at addresses,  add/delete themselves,  and send
mail to the  mailing list.  Commands would be  provided to create
and manipulate  the contents of  a mailing list.   In addition, a
conversion  tool would  be provided  to allow  migration from the
mailing list representation used in MR10.2.


     The  mail system  will be upgraded  to allow a  user to send
mail to another user without  needing to know on which project(s)
the second  user is registered.   In the past,  this facility has
been known as the mail-table.

     The preferred  implementation scheme for  this capability is
to use  the Inquire subsystem  described in MTB-585,  The Multics
Inquire  System:   A  User-Accessible,  User-Maintained, Personal
User  Database  by Barry  Margolin.  However,  if Inquire  is not
available, a  mechanism based on the  present undocumented use of
the >udd>Daemon>mailboxes directory will be implemented.

MTB-608           MR10.2 Mail System Extensions           MTB-608


     The ability  to send mail  without needing to  know a user's
project will manifest itself by a change in the definition of the
-user control argument and the addition of a new control argument
representation of an address.  These new definitions will be:

     -user STR
          specifies  a user's  mail system address.   STR may not
          contain any  whitespace characters; it  may contain, at
          most, one period.  If STR does not contain a period, it
          is  interpreted as  the user's  Person_id and  the mail
          system  address is  obtained from that  user's entry in
          the Inquire database.  Otherwise,  it is interpreted as
          Person_id.Project_id,  the  user's  default  mailbox is
          used, and this control argument is equivalent to:
               -mailbox >udd>Project_id>Person_id>Person_id.mbx

     -last_name STR, -lnm STR
          specifies the mail system address for the user with the
          given last name; the search  of the Inquire database is
          done  in a  case-insensitive manner.  If  more than one
          user  has this  last name,  the user  will be  asked to
          select the desired individual from the list of possible

     A syntax to  allow a search of the  Inquire database by full
name would  be desirable.  However,  such a capability  is beyond
the scope of the current Inquire  subsystem as it would involve a
linear  search  of the  full  name field  which  does not  have a
well-defined  syntax.   If  such  a  facility  is  ever  added to
Inquire, the mail system will use it.

     When sending  mail, if the  user's full name  in the Inquire
database is marked as public,  the mail system will automatically
generate a From  field containing the user's full  name using the
representation described above for -owner.   I.e.:
          From:  Gary M. Palter <Palter>
Of course, if the user supplies an explicit From field, this will
not be done.

     If the command  interface to Inquire is available  only as a
PSP, the  mail system will  supply two command  interfaces to the
Inquire database.   The first command will  display a user's full
name and  mail system address given  their Person_id.  The second
command will allow a user to change his full name and mail system

MTB-608           MR10.2 Mail System Extensions           MTB-608


     If  it  is  not  possible  to  use  Inquire  in  the  MR10.2
time-frame, an alternative mechanism based  on the present use of
>udd>Daemon>mailboxes will be implemented.

     The  directory  >site>mail_system>mailing_addresses  will be
created  with directory  ring brackets of  (2,5).  This directory
will contain one mailing list segment for each user registered on
the system.  These special mailing lists will be known as mailing
addresses.    Each  mailing address  will have  two names  -- the
user's  Person_id  exactly  as  it appears  in  the  PNT  and the
Person_id  translated  to upper-case.   If two  mailing addresses
would have the same all upper-case name, the first user to obtain
a mailing  address would be  given the all  upper-case name.  Any
additional names  (see below) on the  mailing address will always
be in  all upper-case.  The  use of upper-case  only names allows
the  lookup  of  a  user's  mailing  address  to  be  done  in  a
case-insensitive fashion.

     Two  gates   will  be  provided  to   access  these  mailing
addresses.   The  first   gate,  mailing_address_,  is  generally
accessible and  will allow a  user to change the  contents of his
own  mailing  address (creating  it if  necessary) and  will also
permit a user to lookup  another user's mailing address.  Two new
commands,  set_mailing_address and  display_mailing_address, will
be added to allow a user to manipulate his mailing address.

     The  second gate,  mailing_address_admin_, is  restricted to
system  administrators and  will allow the  administrators to add
additional  names  to  mailing   addresses,  create  new  mailing
addresses,  delete  existing  mailing addresses,  and  change the
content  of  any  user's   mailing  address.   Additionally,  the
new_user command  will be upgraded to  automatically initialize a
user's mailing address to their  default mailbox on their default
project when adding a new user.

     The ability  to send mail  without needing to  know a user's
project will manifest itself by a change in the definition of the
-user control argument.  The new definition will be:

     -user STR
          specifies  a  user's  mail   system  address.   If  STR
          contains  exactly one  period and no  whitespace, it is
          interpreted as Person_id.Project_id, the user's default
          mailbox   is  used,   and  this   control  argument  is
          equivalent to:
               -mailbox >udd>Project_id>Person_id>Person_id.mbx
          Otherwise,  the  mailing address  identified by  STR is
          used;  lookup  of  the  mailing address  is  done  in a

MTB-608           MR10.2 Mail System Extensions           MTB-608

          case-insensitive  manner.   The display_mailing_address
          command  can be  used to determine  the mailing address
          corresponding to a given STR.


     As  part of  the ARPA  TCP/IP conversion  effort during late
1982, Charles Hornig implemented  an inter-system mailer known as
mlsys_mailer_.   This mailer  allows a  user to  communicate with
users  on  other computer  systems without  regard to  the actual
networks and  protocols needed to reach  those other systems.  As
part of his implementation, the  Extended Mail Facility and Emacs
RMAIL were upgraded to  use mlsys_mailer_ when communicating with
foreign systems.  However, Executive Mail was not upgraded.

     In MR10.2, mlsys_mailer_ and mail_system_ will be integrated
to form a comprehensive mail system which will allow users of all
three mail PSPs to communicate  with foreign users.  Mail will be
transmitted  between  systems  by   one  or  more  mailer  daemon
processes.  Outgoing  mail will be  placed into a  queue which is
periodically examined by the  mailer daemons.  Incoming mail will
be delivered directly to the user's mailbox by the mailer daemon.
Support will also be provided to allow a Multics system to act as
a  relay  station between  other computer  systems which  have no
direct path of communication.

     In MR10.2,  only the ARPA  TCP/IP SMTP protocol  and the new
inter-Multics mail protocol will be supported.  The SMTP protocol
support will be provided as part of the ARPA TCP/IP PSP (or RPQ).
The inter-Multics  protocol support will  be provided as  part of
the standard system.

     The inter-Multics mail protocol  will operate on either X.25
subchannels or asynchronous communcations channels; however, like
MR10.2 IMFT, reliable delivery is  guaranteed only by use of X.25
subchannels.   The inter-Multics  protocol will  preserve the AIM
access class of the mail  as it is transmitted.  Additionally, it
will support  acknowledgements (send_mail -ack  control argument)
between users on different systems.

     The  command/request  line  syntax  which will  be  used for
foreign  addresses was  presented earlier in  this document.  The
representation of such an address in a message header will be:
          STR {at System1 ...} at SystemN

     The inter-system  mailer requires two  additional facilities
for  proper  operation:   (1)  tasking and  (2)  the network/host
information table.

MTB-608           MR10.2 Mail System Extensions           MTB-608

     The tasking facility provides for multiple execution control
points  within  a single  process.   An overview  of some  of the
issues pertaining to tasking on  Multics may be found in MTB-596,
Tasking I, by  Melanie Weaver and Charles Hornig.   That MTB also
includes  brief  documentation  of a  prototype  tasking facility
developed  by Mr.   Hornig as part  of the  implementation of the
ARPA TCP/IP  project.  This prototype facility  will be installed
as  an  undocumented internal  interface  for use  by  the MR10.2
inter-system mailer.   The facility will  be installed in  such a
way that a process will not  incur any of the additional overhead
of tasking unless it explicitly enables tasking.

     The network/host information table is a database listing all
the networks to which a Multics system is connected, the services
(eg:  mail, remote login) available  on those networks along with
the protocols  used to implement  the services, and  the names of
the   other   systems  on   those  networks.    The  network/host
information  table is  decribed in  MTB-538, Design  of a General
Interface  and Implementation  Structure For Use  in a Networking
Environment, by  Richard Kissel.  A prototype  table format and a
primitive  set of  maitenance functions were  also implemented as
part of the ARPA TCP/IP project.   This prototype will be used by
the  MR10.2   inter-system  mailer.   An   improved  network/host
information  table is  presently being  designed and  will be the
subject of a future MTB.


     The mail_system_ interface presently provides the capability
to add comments to a message when it is forwarded.  The read_mail
forward  request will  be upgraded to  use this  facility via the
addition of a new -add_comments control argument.


     By default, when asked to  add comments, the forward request
will read  the comments from  the terminal.  A  -input_file (-if)
control  argument will  be provided to  allow the  comments to be
read from  a segment.  Like  send_mail, the forward  request will
automatically fill  any comments read from  the terminal but will
not fill comments read from a file.  The standard -fill (-fi) and
-no_fill  (-nfi) control  arguments will be  provided to override
this default.

     If the comments are read from  the terminal and are ended by
a  line  containing  a   period (.),  the  forward  request  will
automatically forward the message.  If, however, the comments are
read from  a file or  terminal input is terminated  via f (which
invokes the qedx editor) or q,  the forward request will enter a
sub-request  loop  in which  the  user may  examine and  edit the

MTB-608           MR10.2 Mail System Extensions           MTB-608

comments  before actually  forwarding the  message.  In addition,
-request_loop   (-rql)   and  -no_request_loop   (-nrql)  control
arguments  will be  available to explicitly  override the default


     As mentioned  above, a sub-request loop  will be implemented
within the  forward request to  allow the user to  edit a comment
before transmission.  The major  requests within this sub-request
loop will be:

     print, pr, p
          prints the comment text.

     print_original, pro
          prints the message(s) which are being forwarded.

     qedx, qx
          edits the comment text using the Multics qedx editor.

     apply, ap
          permits editing  of the comment text  using any Multics

          forwards the message(s) and exits the forward request's
          sub-request loop.

     quit, q
          exits  the forward  request's sub-request  loop without
          forwarding the message(s).

     Unlike the  send request in send_mail,  the send sub-request
listed  above  terminates  the  forward  request  in  addition to
actually  forwarding  the  message.   This  operation  was chosen
because it  is not possible  to change the list  of recipients of
the forwarded message  and, therefore, it does not  make sense to
ask that the message be transmitted more than once.


     When a  message is forwarded,  the mail system  adds several
header  fields  to  indicate  that  this  has  taken  place.   In
addition, if comments are added,  these comments are added to the
message header.  However, as presently implemented, forwarding is
not  a  secure  operation.  In  other  words, it  is  possible to
construct  a  message which  appears  to have  been  forwarded by
someone else.

MTB-608           MR10.2 Mail System Extensions           MTB-608

     In  order  to secure  the  forwarding operation,  the entire
mail_system_  interface  must be  moved to  a lower  ring.  Then,
mail_system_  will  be able  to  verify that  it actually  sent a
message  and  can,  therefore, trust  the  forwarding information
contained therein.  For messages sent  before the securing of the
interface or messages sent from  an outer ring, mail_system_ will
not   trust   the  forwarding   information.   In   these  cases,
mail_system_  will  indicate  to  the  user  that  all forwarding
operations were actually performed by the last person to transmit
the message.

     As  part of  the movement  of mail_system_  to a  lower ring
(ring 2), the  format of messages  stored in the  mailbox will be
changed  to a  binary representation.   This change  will greatly
improve  the  performance of  mail_system_ when  reading messages
from a mailbox.  As part of this conversion, the old mail command
will  be converted  to use  the mail_system_  interface to insure
that it will be able to continue to read the messages stored in a
mailbox.  Additionally,  the Executive Mail  PSP will have  to be
modified to treat all data  returned by mail_system_ as read-only
as said data will actually reside in the lower ring.


     In addition to the major features described above, bugs will
be fixed and minor suggestions will be implemented as an integral
part  of  the  project.   The   exact  list  of  fixed  bugs  and
implemented  suggestions  will be  included with  the appropriate

     Several other  enhancements will be made  to the mail system
if  time permits.   These enhancements include  (in no particular

  o  Acknowledgements of forwarded messages:
          The -acknowledge (-ack) control  argument will be added
          to   read_mail's   forward    request   to   cause   an
          acknowledgement  to be  sent to the  user who forwarded
          the message when  it is read by each  of the recipients
          of the forwarding.

  o  The blind carbon copies (bcc) field:
          The  bcc field  will be supported  in send_mail through
          the use  of the new  -bcc control argument  and the new
          bcc  request.  Recipients  in the  bcc field  receive a
          copy  of  the message  which  indicates that  they were
          blind  recipients  of  the  message.   The  primary and
          secondary  recipients of  the message  are not informed
          that  there are  any blind  recipients of  the message.

MTB-608           MR10.2 Mail System Extensions           MTB-608

          This capability is  already present within mail_system_
          but  additional  work  would  be necessary  to  make it
          available from send_mail.

  o  Secure annotation of messages:
          The user will be permitted  to edit the header and text
          of  a  message already  present in  a mailbox  and then
          rewrite that  message into the mailbox.   The fact that
          the message has been altered is recorded as part of the
          updated message in an unforgeable manner.  This feature
          requires the enhancements to the ring-1 message segment
          primitives  described  in Ned  Kittlitz's  draft MTB --
          >udd>Multics>Kittlitz>mseg>message_segment.mtb       on

  o  Named groups of messages in read_mail:
          A  new  request,  group,  will be  added  to  allow the
          definition  of a  named group of  messages.  This group
          can  then  be used  in later  requests through  the new
          -group control argument.  For example:
               group old_mail -before 10/1/81 -from Palter
                    -subject /mail/
               log -group old_mail -delete

  o  Sending mail to a forum meeting:
          A new type of address will  be added to the mail system
          which specifies that a message should be delivered to a
          specific forum meeting.  This address will be specified
          on command/request lines as:
               -meeting path (-mtg path)
          It will appear in a message header as:
               {forum path}

  o  Unseen messages in read_mail:
          The mail  system will be  extended to record  an unseen
          flag for each message.  This  flag will be reset by the
          read_mail requests print,  reply, write, save, forward,
          append,  and preface  and by  Executive Mail  and Emacs
          RMAIL when they display a message on the terminal.  New
          message specifiers will be  added to read_mail to allow
          the  selection  of  all  unseen  messages  (all_unseen,
          unseen,  au), the  first unseen  message (first_unseen,
          fu),  the next  unseen message  (next_unseen, nu), etc.
          This  capability   is  a  generalization   of  the  new
          transaction specifier in  forum.  Indeed, present forum
          development plans call for the inclusion of transaction
          specifiers similar to  the message specifiers described
          here when the forum meeting format is upgraded to allow
          the bitmap of seen transactions.

MTB-608           MR10.2 Mail System Extensions           MTB-608

  o  New send_mail requests:
          Three     new     requests     --     include_original,
          include_authors,  and  include_recipients  --  will  be
          added  to  send_mail.  These  requests,  available only
          within the send_mail created  by a reply request, allow
          a  user to  insert the  original message  text into the
          reply  or  to  add  the authors  or  recipients  of the
          original  message  as  recipients  to  the  reply after
          reaching  the send_mail  request loop.   These requests
          are     analagous     to     the     -include_original,
          -include_authors,   and   -include_recipients   control
          arguments of the reply request.  The send_mail requests
          allow a  user to still perform  one of these operations
          even  when  they forget  to  use the  appropriate reply
          control arguments.

  o  Conversion of send_mail to use qedx_:
          The qedx  request in send_mail  will be changed  to use
          the new qedx_ subroutine  interface rather than its own
          internal  qedx-like  editor.  This  new version  of the
          qedx  request  will require  the  use of  the write (w)
          request to reflect changes made  to the message back to
          send_mail.   In  addition,  the  quit (q)  request will
          query  if   there  are  modified  buffers   and  a  new
          quit-force (:q, Q) request will be added which does not
          query.   Finally, the  read (r) and  write (w) requests
          will  be  changed  to  not  accept  pathnames  and  the
          read-insert (:r)  and write-copy (:w)  requests will be
          added  to achieve  the old  behavior of  read and write
          with  pathnames.   The send_mail  qedx request  and the
          qedx  command  presently have  a  major incompatibility
          with respect to the behavior of the quit request.  As a
          result, users  who frequently use  send_mail qedx often
          forget  to  save  their  buffers  when  using  the qedx
          command   and  often   lose  large   amounts  of  work.
          Requiring  the   write  request  to   save  changes  in
          send_mail  qedx  is, however,  a  severely imcompatible
          change which can not be made without providing the user
          with some protection; thus, the quit request will query
          when there are modified buffers.

MTB-608           MR10.2 Mail System Extensions           MTB-608


     The following  table presents a  rough estimate of  the time
required to implement each of the enhancements described above:

                        Feature                         Time

            Conversion of Extended Mail to         8 person-weeks
            Conversion of Emacs RMAIL to           4 person-weeks
            Upgrading Executive Mail               4 person-weeks
            Mailing lists                          2 person-weeks
            Mailing by User Name (Inquire)         1 person-week
            Inter-Multics Mail                     6 person-weeks
            Secure Forwarding with Comments        4 person-weeks
               in read_mail
          Total                                   29 person-weeks

            Other Improvements
              Acknowledgements of Forwarded        1 person-day
              The Blind Carbon Copies Field        2 person-days
              Secure Annotation of Messages        1 person-week
              Named Groups of Messages in          1 person-week
              Sending Mail to a Forum              2 person-days
              Unseen Messages in read_mail         1 person-week
              New send_mail requests               1 person-week
              Conversion of send_mail to use       1 person-week
            Total for Other Improvements           6 person-weeks
          Total including Other Improvements      35 person-weeks

     If  the  >site>mail_system>mailing_addresses  mechanism  for
mailing  by  user  name is  to  be  used instead  of  Inquire, an
additional  person-week  should  be  added  to  the  above  total

     The conversion  of Emacs RMAIL  to use mail_system_  will be
performed by Barry Margolin in parallel with my conversion of the
Extended Mail  Facility.  The upgrading of  Executive Mail should
be undertaken  by Dave Schimke  who is the  present maintainer of
that  product  and  should  also  be  done  in  parallel  with my
conversion of  the Extended Mail Facility.   This parallel effort
should reduce the elapsed time for these three tasks from sixteen
to eight weeks.

MTB-608           MR10.2 Mail System Extensions           MTB-608

     If  there  is  not  enough  time  to  implement  all  of the
enhancements   listed   above,  those   items  listed   as  Other
Improvements will be omitted as necessary.