nnadmin - nn database administration

       Kim F. Storm, Texas Instruments A/S, Denmark
       E-mail: storm@texas.dk

4th Berkeley Distribution                          Release 6.6                                       NNADMIN(1m)

       The user interface is completely out of line with the rest of the nn family, and the way to  run  nnadmin
       in  the non-interactive mode is a bit bizarre.  This is not likely to change, because I believe there are
       more important things to do!

       When you select the (S)tat operation in the main or master menus, you will get  some  general  statistics
       about the database:

       initialized
              The time when the database was last rebuild using nnmaster -I.

       last_scan, last_size
              The time stamp on the active file and its size the last time the nnmaster read it.

       no of groups
              The total number of groups in the database.

       Articles
              The  total  number  of articles in all groups.  This is not an exact number, because it will count
              split digests as a single article (making the number too small), and it may  count  some  articles
              that have been expired (making the number too large).

       Disk usage
              The total number of (1 kbyte) disk blocks occupied by the database.

nnadmin  is  a  control  program  for  the  nnmaster(1M)  daemon  which  is  responsible for building and
       maintaining the database used by the nn(1) news reader.

       nnadmin allows you to display extracts from the log file, display the "raw"  contents  of  the  database,
       make  consistency  checks  on  the  database, instruct the running nnmaster to expire one or more groups,
       alter the options of the running nnmaster, and much more.

       nnadmin runs in two modes: interactive and non-interactive.

       In interactive mode, simple one line menus are used to show  the  available  operations  which  are  then
       selected  by  typing  the  letter  associated  with the command (normally the first letter in the command
       name).

       In non-interactive mode, the commands argument will  be  used  as  a  series  of  key-strokes  which  are
       interpreted exactly as if they were typed in from the keyboard in interactive mode.  For example, to stop
       the nnmaster, the following invocation of nnadmin can be used:
            nnadmin MK
       which will select the (M)aster submenu from the main menu, and then the (K)ill entry from the submenu.

       In  non-interactive mode, the menus are not displayed and the commands are not echoed!  nnadmin will exit
       when there are no more key-strokes to be read from the commands argument.  It is not possible to  specify
       a  group name in the commands argument, so the functionalities of nnadmin that relates to specific groups
       are only available in interactive mode.

       Some "dangerous" commands will require that you confirm them by following them  by  "Y"  on  the  command
       line.  The most noteable are IY (initialize database) and EY (expire all groups).  These commands will be
       marked with a [Y] following the command name.

       You can also invoke an interactive nnadmin using the :admin command in nn.

       The  $db,  $lib,  and  $news  used  below  are synonyms for the DB_DIRECTORY, LIB_DIRECTORY, and the news
       system's lib directories respectively.
       $db/MASTER        Database master index
       $db/GROUPS        News group names in MASTER file order
       $db/DATA/nnn.x    Index file for group number nnn
       $db/DATA/nnn.d    Data file for group number nnn
       $master/GATE      Message channel from nnadmin to nnmaster
       $master/MPID      The process id of the nnmaster daemon.
       $Log              The log file (truncate it regularly!)

       The MASTER file contains a record for each news group, occurring in the same sequence as the group  names
       in  the  GROUPS  file.   The  sequence  also  defines the group numbers used to identify the files in the
       database and in a few other places.

       The GATE file will be created by nnadmin when needed, and removed  by  nnmaster  when  it  has  read  it.
       Therefore,  to  send  a  message  to  the  nnmaster requires that you are allowed to write in the $master
       directory.

       You  can  set  and  clear  the  following  flags for individual groups to control the future behaviour of
       nnmaster on that group.

       Notice that these flags will be reset to their default value  if  you  reinitialize  the  database  using
       nnmaster -I.  To change these flags permanently, they should be set or cleared in the GROUPS file.

       A)lways_digest
              Normally,  nnmaster  will  only attempt to split digests into individual articles if it can easily
              recognize an article as a digest.  This requires that the word "digest" appears somewhere  in  the
              subject  line,  and that one of the first few lines in the body of the article loosely matches the
              subject.  A few news groups  frequently  receives  digests  which  break  one  or  both  of  these
              requirements.   To have nnmaster split these digests into individual articles anyway, you can turn
              on the "always digest" flag on these news groups.   This  will  instruct  nnmaster  to  treat  all
              articles  in  the  group as digests (naturally, articles which are then found not to contain other
              articles are still treated as normal articles.)

       C)ontrol
              This is a special flag for the control group.  It indicates that the "Newsgroups:"  field  in  the
              article  header  cannot  be  trusted (it does not specify the groups to which the article has been
              posted.)

       D)irectorymissing
              This flag indicates that the spool directory for the news group cannot be  found  (the  group  has
              probably  been  removed  with  rmgroup(1M)).  It is set automatically be the nnmaster if it cannot
              access the directory.  When the flag is set, nnmaster completely ignores the group, so it  can  be
              used  to  disable  news collection in specific groups.  If you recreate the group or the directory
              manually, you must also clear this flag to have the nnmaster recognize the group again.

       M)oderated
              Indicates that the group is moderated.  This flag is normally initialized automatically  from  the
              active file, and it should not be changed lightly.

       N)ever_digest
              This  is  the  opposite  of the "always digest" flag; when set, the nnmaster will never attempt to
              split any articles in that group into subarticles.

       From the main menu (identified by the ADMIN prompt) you can select the following operations:

       C)onf
              Show current configuration parameters such as directories, files, programs, network usage, etc.

       E)xpire[Y]
              Send a request to the nnmaster daemon to schedule (and run) expire for all groups in the database.

       G)roups
              Enter the GROUP submenu.

       I)nit[Y]
              Send a request to the nnmaster daemon to recollect all groups in the database.

       L)og
              Enter the LOG submenu.

       M)aster
              Enter the MASTER submenu.

       Q)uit
              Quit nnadmin.

       S)tat
              Print general statistics about the database.  See the section on Database Statistics below.

       U)pdate
              Update the incore copy of the database master index.

       V)alidate
              Make a thorough consistency check on the database.  If inconsistencies are found in a  group,  you
              will  be  asked whether a request should be sent to the nnmaster daemon to recollect the group (in
              non-interactive mode, requests will be sent automatically for all corrupted groups).

       W)akeup
              Send a wakeup signal to the nnmaster daemon to have it receive messages sent to  it,  perform  the
              required actions, and then collect articles as necessary.

       Z (silent validation)
              This  operation  is  identical to the Validate operation, expect that no output is produced during
              the consistency check; this operation is used by the nnmaster to execute the -C option.

       The  master  index entries displayed when you select the (H)eader operation in the master and group menus
       contain the following information:

       group_namegroup_number
              The first line of the display will show the name of the group and the internal group number  which
              is used to identify the group in the database.

       first/last art
              This is the numbers of the first and last article that are currently stored in the database.

       active info
              This is the numbers of the first and last article in the news system as read from the active file.
              They  will  normally match the numbers above, but they may differ while the nnmaster is working on
              the group (or it has not yet collected all the articles in the group).

       Offsets: index->..., data->...
              These values show the starting position for the next write operation on the index and data  files.
              They  are  primarily used for consistency checking and recovery after a system crash, but after an
              "expire by rewrite" operation (expire method 2) which is performed "in-situ", the data  and  index
              files may physically be longer than the actual data stored in them.

       Flags:
              This  shows  the current flags set for this group.  If no flags are set, the field is omitted from
              the display.  One extra flag which was not explained above is the BLOCKED flag; it is a  temporary
              locking flag set on a group by the nnmaster while it is updating the database files for that group
              to prevent nn clients to access that group.

       nnadmin - nn database administration

       When  you  select the (D)ata operation on the group menu, you will get a combined display of the raw data
       and index files for that group.  The index file contains a single 32 bit value for each existing  article
       number.  This value is an offset into the data file pointing to the header for the corresponding article.

       When  nn want to access the article from number N to the last article, it looks up the offset for article
       number N in the index file, and uses this as the starting point for reading article header information in
       the data file.  It then simply reads to the end of the  data  file  in  which  the  article  headers  for
       articles number N+1, N+2, and so on follows immediately after the header for article number N.

       The  article header information is presented in a very terse form; each of the output lines are described
       below for reference purposes:

       offset = xxxx    , article # = nnnnn   (type)
              This shows the offset into the data file and the article number.  The  offset  is  stored  in  the
              index  file  for  quick  access.   If no type is printed it is a normal article.  Other types are:
              "digest header" and "digest sub-article".

       xpost(count):  nnn, nnn, nnn, ...
              Cross-postings to other groups are encoded as a list of internal group numbers.

       ts=nn hp=nn fp=nn lp=nn ref=nn[+Re] lines=nn
              These values are used by nn to sort, present, and access an article:
              ts is the timestamp on the article; it is a simple encoding of the posting date and time found in
              the Date: field.
              hp, fp, and lp are offsets into the file containing the article text: the headerposition,  firsttextposition,  and  lasttextposition.  The first will be zero for normal articles, but not for
              articles in a split digest.  The last will be equal to the length of the file for normal articles,
              but not inside digests.
              ref is the number of references on the Reference: line.  If "+Re" follows the number, the  subject
              line contained a "Re:" prefix which has been removed.

       Sender(length): name
              The  name  of the sender in "ready to print" format, i.e. reduced to 16 characters as explained in
              the nn manual.

       Subj(length): subject
              This is the full subject line from  the  article  header  (except  for  Re:  prefixes  in  various
              formats).

nn(1), nncheck(1), nngrep(1), nntidy(1)
       nnquery(1M), nnusage(1M), nnmaster(8)

       At all prompts you can hit `!' to spawn a subshell.

       The  working  directory  of  the subshell will be changed to the database directory when invoked from the
       MASTER or DUMP menus, and it will changed to the group's spool directory (if it exists) when invoked from
       the GROUP menu.

nnadmin [ commands ]

       The  dump  menu  (identified  by  the DUMP prompt) allows you to print the master index entry for various
       selections of groups in the database.

       A)ll
              Print all groups in the database.

       E)mpty
              Print the empty groups in the database.

       H)oles Print the groups where the `min' field in the active file is not the first article  saved  in  the
              database  (because  it  doesn't  exist or because it is ignored for some other reason, e.g. bad or
              old).

       I)gnored
              Print groups which are ignored, either in the GROUPS file  or  because  of  some  other  condition
              (mainly no spool directory).

       N)on-empty
              Print the non-empty groups in the database.

       V)alid Print the groups which are present in the active file.

       in(W)alid
              Print the groups in the database which are not present in the active file.

       When you enter the group menu (identified by the GROUP prompt), nnadmin will prompt you for the name of a
       news group, which you can enter with the usual completion feature described in the nn(1) manual.  You can
       then perform the following operations on the specified group:

       C)lear_flag
              Clear a group specific flag.  See the section on group flags below.

       D)ata
              Dump the contents of the data file containing the extracted article headers for the group.

       E)xpire
              Request the nnmaster to run expire on the group.

       F)iles
              List the files (using ls(1)) containing the index and data for the group.

       G)roup
              Switch to another group.

       H)eader
              Dump the master index entry for the group.

       R)ecollect
              Request the nnmaster to recollect all articles in the group.

       S)et_flag
              Set a group specific flag.  See the section on group flags below.

       V)alidate
              Perform validation on the group's database information.

       Z)ap[Y]
              Remove group from news system - this will be done by running the rmgroup program which must reside
              in the NEWS_LIB directory.  Of course, this should be done with great caution.

       The  log  menu (identified by the LOG prompt) enables you the extract specific entries from the log file,
       and to truncate the log file.

       The entries in the log file share the following format:
            <class>: <date> <time> (<user>): <message>
       where <class> identifies the message class, the <date> and <time> specify when the entry  was  made,  the
       <user>  specifies  who  created  the entry (the letter "M" denote the nnmaster), and the <message> is the
       text of the entry.

       To extract the log file entries of a specific class, simply enter the letter identifying the class:

       A-admintomastercommunication
              This class of messages are related to the sending of messages  from  an  nnadmin  program  to  the
              nnmaster daemon.

       B-badarticles
              Reports about bad articles which have been ignored or removed (controlled by the -b and -B options
              to nnmaster).

       C-collectionstatistics
              Statistics about collection of new articles.  The message has the format:
                   Collect: nnn art, ppp gr, ttt s
              meaning that nnn articles in ppp groups were collected in ttt seconds (real time).

       E-fatalerrors
              Fatal  errors  encountered during operation.  These errors require manual intervention to be fixed
              (some of the fatal errors occur if thing that "cannot happen" happens anyway, and may  indicate  a
              bug in the software).

       M-nnmastermessages.
              Master start/stop messages.

       N-NNTPrelatedmessages
              Various  messages  related  to  the  NNTP  part of the nnmaster, mostly about lost connections and
              failed attempts to connect to the NNTP server.  These messages should only appear if you use NNTP,
              and your NNTP server is down for some reason.

       O-oldarticles
              Reports related to ignoring (and removing) old articles when building the database (controlled  by
              the -O and -B options to nnmaster).

       R-reports
              Non-fatal error which enables the nnmaster to continue operation, but may prevent a user to run nn
              (file access problems).  Reported problems should be checked.  The most common report message will
              probably be
                   some.group: no directory
              which  indicates  that  the spool directory for that group has disappeared (most likely because it
              has been rmgroup'ed).

       T-traceoutput
              Messages produced as a result of using the -t option on  the  nnmaster.   This  is  primarily  for
              debugging purposes.

       U-usagestatistics
              If  nn is compiled with the STATISTICS option enabled, an entry will be made in the log file every
              time a user has spent more than five minutes on news reading.  The message will have the following
              format:
                   USAGE hours.minutes
              Since it is possible to suspend nn, or leave the terminal while nn  is  active,  nn  tries  to  be
              intelligent  when  it  calculates  the usage time so it will reflect the actual time spent on news
              reading.  The usage statistics can be summarized using the nnusage(1M) program.

       V-validationerrors
              When inconsistencies are detected in the database during validation, an entry for  each  corrupted
              group will be entered in the log file.

       X-expirestatistics
              Messages similar to the Collect statistics reporting the result of running expire on the database.
              Reports  related  to  ignoring,  removing,  renumbering, and reactivation of groups are also given
              class X.

       To extract a specific entry class, grep(1) is used, so it may take a while on a large log file.

       There are also a few special operations on the log file:

       G)roup
              Extract the entries which refers to a specified group.

       (1-9)tail
              Invoke tail(1) to extract the last 10-90 entries in the log file.

       space
              Equivalent to 1 (list last 10 lines of log).

       (.)all
              Display the complete log file.

       (@)clean[Y]
              Move the Log file to Log.old, and create a new empty Log file.  If you want to clean out  the  old
              log file as well, simply repeat the clean operation (this will result in an empty Log.old file.)

       The master menu (identified by the MASTER prompt) provides access to overall database information, and to
       send control messages to the nnmaster daemon.

       C)heck In interactive mode and in verbose batch mode  (nnadmin  MC),  print  a  message  telling  whether
              nnmaster  is  running  or not.  In silent batch mode (nnadmin =MC) exit with a status code of 0 if
              nnmaster is running and 1 otherwise; this may be useful is administrative scripts.

       D)ump  Enter the DUMP submenu.

       F)iles
              Print a listing (using ls(1)) of all the data and index files in the database.

       G)roup
              Print the master index entry for a single group identified by its internal group number.

       K)ill
              Stop the nnmaster when it has finished its current task.

       O)ptions
              Change the runtime options of the running nnmaster daemon.  Currently, only the value  of  the  -r
              and -e options can be modified.

       S)tat
              Print general statistics about the database.  See the section on Database Statistics below.

       T)race
              Turn the trace option -t on or off in the running nnmaster.

       The GATE file is created with the owner and modes of the user that runs nnadmin which may cause  problems
       if  the  owner  of the nnmaster process (normally "news") is not allowed to read the created GATE file (a
       "umask" of 022 is ok.)  Unless you allow ordinary users to create files in the LIB  directory  where  the
       GATE  file  resides, only the owner of the directory (normally "news") and "root" can use nnadmin to send
       messages to the nnmaster.  However, to send a wakeup signal to the master, anybody can run
            nnmaster -w