To:       MAB Distribution

From:     Gary C. Dixon
          John W. Gintell
          R. C. vonSeeburg

Date:     February 13, 1981

Subject:  Revised MAB on TR Goals


This MAB describes changes to MAB-044, which defines  Multics  TR
Processing  Goals.   Most  significant  changes  in definition of
goals include the following:

(1) On page 3 in discussion of response, the  following  sentence
    was removed.

       Response  time  includes  the  time  needed  to  route the
       response back to the submitter.

    The TR System does not include such time.  Perhaps it should,
    but it doesn't.

(2) Removal of concept of installed-in-EXL being a resolution  to
    a problem.  This decision was made some time ago, but the MAB
    was never updated to reflect the policy change.

(3) The TR goal is stated in terms of  completing  processing  of
    75% of TRs within the expected amount of time (times given in
    Table  1).   The  prior  MAB  did  not  state  this 75% goal,
    implying our goal was to process the average  TR  within  the
    expected time.

    The  goal  is  measured as percent_completed_on_time for each
    type and processing stage, weighted by the number of  reports
    completed in that stage.  The exact algorithm is given in the

All  other  changes  are minor wording changes or clarifications.
The intent of this revision is to reflect  the  current  policies
and procedures now in effect.


To:       MAB Distribution

From:     Gary C. Dixon
          John W. Gintell
          R. C. Von Seeburg

Date:     February 4, 1981

Subject:  Goals for Processing Multics Trouble Reports


     This  bulletin  defines  performance goals for processing of
Trouble Reports (TRs) entered  by  Multics  users  through  their
SiteSAs, and categorized in the TR System as external TRs.


     The  criteria  for  measuring  achievement  of TR processing
goals are defined using the following terminology.

Types of TRs

There are three kinds of trouble reports.

     1.  Problem Reports: reports of problems in installed system
         programs, info segments, or system documentation.

     2.  Questions: questions about operation of the  system,  or
         about  installed  system  programs,  info  segments  and

     3.  Suggestions:  suggestions  for  changes/improvements  to
         installed  system  programs,  info  segments,  or system


Multics  Project  internal  working  documentation.   Not  to  be
reproduced or distributed outside the Multics Project.

Problem Report Priorities

     Problem reports are further divided  into  Priority  groups,
according to the significance of the problem.

     1.  Problem Reports

         A.  Critical Priority, a problem which:

             -  interferes with system  operation  to  an  extent
                that  usefulness  of  the system is significantly
                limited for most users of the system. (1)

(1) This includes, but is not limited to, situations in  which  a
site  is  totally down, or is running in a highly crippled state.

             -  irreparably compromises the security or integrity
                of data  stored  in  the  system.   Data  storage
                problems  at  system  priority  must  affect many
                system files, or files  which  are  difficult  to
                retrieve or recreate.

         B.  High Priority, a problem which:

             -  causes  installed  system  programs  to  fail  to
                operate  correctly in limited circumstances, with
                no known bypass for the problem.   Such  problems
                can be tolerated for short periods of time.

             -  compromises data  security  or  integrity  for  a
                small  number  of  files  in  a  way which can be
                repaired without significant loss of data.   Such
                problems  can  be  tolerated for short periods of

         C.  Normal Priority, a problem which:

             -  causes installed system programs  to  fail  in  a
                minor way which can be bypassed, or ignored.

             -  results from incorrect documentation.

TR Processing Operations

     Processing of TRs involves the following steps.

     1.  Submission occurs when a SiteSA or user enters a problem
         report, suggestion, or question.  A report is entered by
         using  the  enter_problem_report (epr), enter_suggestion
         (es), or enter_question (eq) command on System M.

         A SiteSA usually reports Critical  problems  to  Multics
         Software  Support  (MSS)  by telephone to avoid delay in
         processing.   For  performance  tracking  purposes,  MSS
         personnel  enter  such telephoned TRs into the TR system
         after the initial telephone conversation is complete.

     2.  Verification of a report validates the user's  statement
         of  problem,  or  initially  appraises  suggestions,  or
         answers questions when answer is known.  Verification is
         performed by the TR  Administrator  or  investigator  in

     3.  Routing of a report forwards the report  to  developers,
         investigators,   and  interested  parties.   Routing  is
         performed by the TR Administrator in Phoenix.

     4.  Response to a problem report: acknowledges existence  of
         a  bug,  indicates  a bug bypass if any, and states that
         problem will be  fixed  in  a  future  release;   or  it
         indicates  that  the problem cannot or will not be fixed
         (a system limitation).

         No  response  is  required for questions or suggestions.

(2) The answer to a question is provided when the question report
is resolved.  Current management policy states that response  to,
and resolution of, suggestions by the developer is optional.

         Response is made by the developer,  investigator  or  TR
         Administrator,  and is forwarded to the submitter by the
         TR Administrator.  A response is entered  by  using  the
         answer_trouble_report (atr) command.

     5.  Resolution of a problem report occurs  when  changes  to
         installed   system  programs  causing  the  problem  are
         submitted  for  installation  in  the   Multics   System
         Libraries,  or  when documentation errors are corrected;
         or when the nature of a user error causing  the  problem
         is  pointed  out to the submitter; or when the report is
         unclear or  unreproducible  and  additional  information
         about the problem is required.

         Resolution of a question report occurs when the question
         is answered.

         Software  problems  are  considered  resolved  when  the
         programs  are  submitted  for  installation  in   System
         Libraries  on  System  M,  because  at  this  point,  an
         emergency fix can be shipped to  the  site  if  this  is
         deemed   necessary.    Bug   fixes   are   not  normally
         distributed between releases however, and the time  from
         submission of installation of software on System M until
         distribution  of  the  bug  fix  is  NOT included in the
         resolution time.

         Documentation is considered  fixed  when  the  corrected
         manual is sent to the Brighton Print Shop.

         A  trouble  report  resolution  is  entered by using the
         answer_trouble_report  (atr)  command.   Developers  are
         responsible  to entering notification that a problem fix
         has been submitted  for  installation  into  the  System
         Libraries.     The    Manager   of   Multics   Technical
         Documentation at CISL and the  Project  Leader  for  the
         Multics Documentation Project in Phoenix are responsible
         for   entering  final  resolution  notice  when  manuals
         containing corrections are sent to  the  Brighton  Print


     The  goal  for  TR processing performance states that 75% of
all TR processing operations will be completed within a specified
amount of time, where times are measured from submission  of  the
TR  into  the  TR  system  to  completion  of  the  TR processing

     The procedures for TR processing state the  order  in  which
TRs  are  to be processed, and the expected amount of time needed
to perform each TR processing operation.

Expected TR Processing Times

     The expected amount of time  required  to  perform  each  TR
processing  operation  is  shown  in  the table below.  Times are
expressed as the number of calendar  weeks  which  should  elapse
from  time  of  submission  to  the time of operation completion,
averaged over all TRs whose operations have completed during  the
reporting  period.   Because  the times are averages, they do not
state the processing performance expected for  a  particular  TR.
They instead address our overall TR processing performance.

        Table 1:  Average Calendar Weeks from Submission
                  to Completion of TR Processing Operation

OPERATION          Problem              Problem
COMPLETED      Critical  High   Query   Normal      Suggestion
               |                                                |
Verification   |   0.2     1       1        2            4      |
               |                                                |
Response       |   1       2       4       13           --      |
               |                                                |
Resolution     |   4      13       4       52           --      |

     Verification  and  routing  operations  are  performed  as a
single processing operation, and are therefore shown as a  single
processing goal in the table above.

     Also,  as  the  table indicates, no response is required for
questions.  No response or resolution is required for  suggestion
reports since current management policies state that response and
resolution of suggestions is optional.

Formula For Computing Performance To Goal

             _____                 _____
             \                     \     # ops completed
total_ops =   >                     >    since 07/01/80
             /____                 /____
        for problems (critical,  for verification,
        high & normal) and       response and
        questions                resolution ops

             _____                 _____
             \                     \     # ops           %_on_time
%_on_time =   >                     >    completed     * for op
             /____                 /____ since 07/01/80
        for problems (critical,  for verification,
        high & normal) and       response and
        questions                resolution ops

%_performance = %_on_time / total_ops

Our goal is:  %_performance >= 75%

Order of Processing TRs

     TRs   should   be   processed  in  the  following  order  of

     1.  Problem Reports, Critical Priority

     2.  Problem Reports, High Priority

     3.  Questions

     4.  Problem Reports, Normal Priority

     5.  Suggestions

Type of report and priority  of  problem  reports  are  initially
specified  by the submitter of the report, but can be adjusted by
investigators, TR Administrator (who routes TRs)  and  developers
as the report becomes better understood.


     TR  Processing  performance will be reported to the Director
of Software  Development  on  a  quarterly  basis.   It  will  be
reported weekly to the Manager of Multics Software Development in
status reports.  Reporting will begin with statistics for reports
submitted during the third quarter, 1979.