ACTSYNC (8)
synchronize newsgroups
SYNOPSIS
actsync
[-b hostid] [-d hostid] [-g max]
[-i ignore_file]
[-I hostid] [-k] [-l hostid] [-m] [-n name]
[-o fmt] [-p min_%_unchg] [-q hostid] [-s size]
[-s spool_dir] [-t hostid] [-T] [-v verbose_lvl]
[-z sec] [host1] host2
actsyncd
[-x] actsync.cfg [debug_level [debug_outfmt] ]
DESCRIPTION
Actsync (8)
permits one to synchronize, compare or merge two
active (5)
files.
With this utility one may add, change or remove newsgroups on the
local news server to make it similar to the list the newsgroups
found on another system or file.
The synchronization need not be exact.
Local differences in newsgroup lists may be maintained and preserved.
Certain newsgroup errors may be detected and optionally corrected.
There are several reasons to run
actsync (8)
(or
actsyncd (8)),
on a periodic basis.
Among the reasons are:
+0.5i
A control message to add, change or remove a newsgroup
may fail to reach your site.
Your
control.ctl (5)
is out of date or incomplete.
News articles for a new newsgroup arrive ahead (sometimes days ahead)
of the control message.
Control messages may be forged, thus bypassing the restrictions
found in
control.ctl (5).
Your
active (5)
file may have been trashed.
-0.5i
If either
host1
or
host2
begin with a
``.''
or
``/'' ,
then they assumed to be a name of a file containing information in the
active (5)
format.
The
getlist (1)
utility may be used to obtain copy a remote system's active file
via its NNTP server, or an FTP client program can get retrieve such a
file from an FTP archive (such as
ftp://ftp.isc.org/pub/usenet/CONFIG/active; see more about this below).
Newsgroup information from a file
may be treated as if it was obtained from a host.
In this man page
host1
and
host2
are called hosts, even though they may be file names.
If a host argument does not begin with
``.''
or
``/'' ,
is assumed to be a
hostname or Internet address.
In this case,
actsync (8)
will attempt to use the
NNTP
protocol to obtain a copy of the the specified system's active file.
Regardless how the active file information is obtained,
the actions of
actsync (8)
remain the same.
If only one host is specified, it is assumed to be
host2 .
If
host1 ,
is not specified, it assumed to be the default local
NNTP server as specified by the
NNTPSERVER
environment variable, or by the
server
value found in
inn.conf (5).
The newsgroup synchronization by default, involves all newsgroups
found on both hosts.
One may also synchronize on a subset of newsgroups by directing
actsync (8)
to ignore certain newsgroups from both systems.
The
actsyncd (8)
daemon provides a convenient interface to configure and run
actsync (8).
If a host is not initially reachable,
the daemon will thrice retry 9 times, waiting 6 minutes before
each retry.
This daemon runs in the foreground, sending output to standard output
and standard error.
If the -x flag is given to
actsyncd (8),
then a
ctlinnd xexec
will be used instead of a
ctlinnd reload
to load the newly modified active file.
The configuration filename for the daemon is given in the
actsync.cfg
argument.
The
actsync.cfg
file understands the following options:
+0.5i
host=host2
ftppath=/remote/path/to/active/file
spool=<normally patharticles in inn.conf>
ignore_file=ignore_file
flags=actsyncd (8) options
-0.5i
The host, ignore_file and flags lines are mandatory.
The keyword must start at the beginning of the line, and there
may be no whitespace before the
``=''
character.
Blank lines are ignored.
Comments start with
``#''
and are ignored.
All other lines may produce undefined results.
The host config file line refers to the host2 value to sync off of.
the ftppath directive causes the machine named in the host
line to accessed as an ftp server, retrieving the file named. If
the filename ends in .gz or .Z, then it will automatically
be uncompressed after retrieval.
The spool config file lines determines where top the
news spool tree is to be found.
The ignore_file config file line names the ignore file to be
used by
actsync (8).
The flags config file line refers to all flags that you wish to pass to
actsync (8).
Note that the -i ignore_file option,
the -o format option
and the -S spool_dir option
should not be given
in the flags= line because they are automatically taken care of by
actsyncd (8).
INN is shipped with default values of ftp.isc.org for host
and /pub/usenet/CONFIG/active for ftppath. You can read
about the policies used for maintaining that active file at
ftp://ftp.isc.org/pub/usenet/CONFIG/README. Consider
sychronizing from this file on a daily basis by using cron.
OPTIONS
The options to
actsync (8)
are as follows:
-b hostid
This flag causes
actsync (8)
to ignore newsgroups with ``bork.bork.bork'' style names.
That is, newsgroups whose last 3 components are identical.
For example, the following newsgroups have bork style names:
+0.5i
alt.helms.dork.dork.dork
alt.auto.accident.sue.sue.sue
alt.election.vote.vote.vote
-0.5i
The value
hostid
determines on which hosts this action is performed:
+0.5i
0 neither host
1 local default server
2 remove server
12 both servers
21 both servers
-0.5i
The default is
-b 0
no bork newsgroups are ignored.
-d hostid
This flag causes
actsync (8)
to ignore newsgroups that have all numeric path components.
The
hostid
value is interpreted the same as in
-b .
For example, the following newsgroups have numeric path components:
+0.5i
alt.prime.chongo.23209
391581.times.2.to_the.216193.power.-1
99.bottles.of.treacle.on.the.wall
linfield.class.envio_bio.101.d
-0.5i
The newsgroups directory of a newsgroups with a all numeric component
could conflict with an article from another group.
For example, the directory for the first newsgroup listed above
is the same path as article number 23209 from the newsgroup:
+0.5i
alt.prime.chongo
-0.5i
The default is
-d 0
all numeric newsgroups from both hosts will be processed.
-g max
Ignore any newsgroup with more than
max
levels. For example,
-g 6
would ignore:
+0.5i
alt.feinstien.votes.to.trash.freedom.of.speech
alt.senator.exon.enemy.of.the.internet
alt.crypto.export.laws.dumb.dumb.dumb
-0.5i
but would not ignore:
+0.5i
alt.feinstien.acts.like.a.republican
alt.exon.admendment
alt.crypto.export.laws
-0.5i
If
max
is 0, then the max level feature is disabled.
By default,
the max level feature is disabled.
-i ignore_file
The
ignore_file
allows one to have a fine degree of control over which newsgroups are ignored.
It contains a set of rules that specifies
which newsgroups will be checked and which will be ignored.
By default, these rules apply to both hosts.
This can be modified by using the
-I hostid
flag.
By default, all newsgroups are checked.
If no
ignore_file
if specified, or if the ignore file contains no rule lines,
all newsgroups will be checked.
Blank lines, and text after a
``#''
are considered comments and are ignored.
Rule lines consist of tokens separated by whitespace.
Rule lines may be one of two forms:
+0.5i
c newsgroup [type ...]
i newsgroup [type ...]
-0.5i
If the rule begins with a
c
then the rule requests certain newsgroups to be checked.
If the rule begins with an
i
then the rule requests certain newsgroups to be ignored.
The
newsgroup
field may be a specific newsgroup, or a
wildmat (3)
pattern.
If one or more
type s
are specified, then the rule applies to the newsgroup only if
is of the specified type.
Types refer to the 4th field of the
active (5)
file.
A type may be one of:
+0.5i
y
n
m
j
x
=group.name
-0.5i
Unlike active files, the
group.name
may be a newsgroup name or a
wildmat (3)
pattern.
Also,
``=''
is equivalent to
``=*'' .
For given rule line may, one may not repeat a given pattern type.
For example, one may not have more than one type that begins with
``='' ,
per line.
However, one may achieve the effect of multiple
``=''
types by using multiple rule lines for the same group.
By default, all newsgroups are candidates to be checked.
If an ignore file is used, each newsgroup in turn is checked
against the ignore file.
If multiple lines match a given newsgroup, the last line
in the ignore file is used.
For example, consider the following ignore file lines:
+0.5i
i *.general
c *.general m
i nsa.general
-0.5i
The newsgroup:
ba.general
would be ignored if it was not moderated.
The newsgroup:
mod.general
would be checked if it was moderated.
The newsgroup:
nsa.general
would be ignored even if it was moderated.
-I hostid
This flag restricts which hosts, the ignore file applies.
The
hostid
value is interpreted the same as in
-b .
This flag may be useful in conjunction with the
-m
merge flag.
For example:
+0.5i
actsync -i actsync.ign -I 2 -m host1 host2
-0.5i
will keep all newsgroups currently on host1.
It will also will only compare host1 groups with
non-ignored newsgroups from host2.
The default is
-I 12
newsgroups from both hosts to be ignored per the
-I " hostid"
flag.
-k
By default, any newsgroup on
host1
that is in error will be considered for removal.
This causes
actsync (8)
simply ignore such newsgroups.
This flag, in combination with
-m
will prevent any newsgroup from being scheduled for removal.
-l hostid
Flag problem newsgroups of type
``=''
from
host1
or
host2
as errors.
The
hostid
value is interpreted the same as in
-b .
Newsgroups of type
``=''
are newsgroups active entries that have 4th field
that begins with
``='' .
I.e., a newsgroup that is equivalent to another newsgroup.
A newsgroup that is equivalent to itself, or that is in a equivalence
chain that loops around to itself is a problem.
A newsgroup that is in a chain that is longer than
16
is a problem group.
A newsgroup that is equivalent to a non-existent newsgroup is a problem.
A newsgroup that is equivalent to a newsgroup that is has a error
of some kind a problem.
However, a newsgroup that is equivalent to an ignored newsgroup is
not a problem.
By default, problem newsgroups from both hosts are
marked as errors.
-m
Merge newsgroups instead of sync.
By default, if a newsgroup exists on
host1
but not
host2 ,
it will be scheduled to be removed.
This flag disables this process, permitting newsgroups unique to
host1
to be kept.
-n " name"
Newsgroups that are created, are created via the
ctlinnd (8)
command.
By default, the creator name used is
actsync .
This flag changes the creator name to
name .
-o " fmt"
Determine the output / action format of this utility.
The
fmt
may one of:
+0.5i
a output in active(5) format,
a1 output in active(5) format,
and output host1 non-error ignored groups
ak output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for any newsgroup being created
aK output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for all newsgroups found in host2
a1k output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for any newsgroup being created,
and output host1 non-error ignored groups
a1K output in active(5) format, but use host2
hi & low (2nd & 3rd active fields) values
for all newsgroups found in host2,
and output host1 non-error ignored groups
ak1 same as a1k
aK1 same as a1K
c output in ctlinnd(8) format
x no output, directly exec ctlinnd(8) commands
xi no output, directly exec ctlinnd(8) commands,
in an interactive mode
-0.5i
The a, a1, ak, aK, a1k,
a1K, ak1 and aK1 style formats allow one to form
a new active file instead of producing
ctlinnd (8)
commands.
They use hi & low values of
0000000000
and
0000000001
respectively for newsgroups that are created.
The ak and aK variants change the the hi & low (2nd & 3rd
active fields).
In the case of ak, newsgroups created take their hi & low values from
host2 .
In the case of aK, all newsgroups found on host2 take their
hi & low values from
host2 .
The c format produces
ctlinnd (8)
commands.
No actions are taken because
actsync (8)
simply prints
ctlinnd (8)
commands on standard output.
The sync (or merge if -m) with
host2
may be accomplished by piping this output into
sh (1).
A paranoid person might prefer to use x or xi
in case a newsgroup name or type contains bogus characters
that might be interpreted by
sh (1).
Even so, this output format is useful to let you see how
host1
may be synced (or merge) with
host2 .
The sync (or merge if -m) may be accomplished directly
by use of the x.
With this format,
actsync (8)
uses the
execl (2)
system call to directly executes
ctlinnd (8)
commands.
Because of the exec, there is no risk
of bogus newsgroups containing bogus characters causing
a shell to do bogus (or dangerous) things.
The output of such execs may be seen of the verbosity level
is at least
2 .
The
actsync (8)
utility will pause for
4
seconds before each command is executed if
-o x
is selected.
See the
-z sec
flag below.
The xi format interactively prompts on standard output
and reads directives on standard input.
One may pick and choose changes using this format.
Care should be taken when producing
active(5) formatted output.
One should check to be sure that
actsync (8)
exited with a zero status prior to using such output.
Also one should realize that such output will not
contain lines ignored by the
-i ignore_file
process even if
-p 100
is used.
By default,
-o c
is assumed.
-p min_%_unchg
By default, the
actsync (8)
utility has safeguards against performing massive changes.
If fewer than
min_%_unchg
percent of the non-ignored lines from
host1
remain unchanged, no actions (output, execution, etc.)
are performed and
actsync (8)
exits with a non-zero exit status.
The
min_%_unchg
may be a floating point value such as
66.666 .
A change is considered a
host1
line that was found to be in error,
was removed, was added or was changed.
Changing the 2nd or 3rd active fields via
-o ak
or
-o aK
are not considered changes by
-p .
To force
actsync (8)
to accept any amount of change, use the
-p 0
option.
To force
actsync (8)
to reject any changes, use the
-p 100
option.
Care should be taken when producing
active(5) formatted output.
One should check to be sure that
actsync (8)
exited with a zero status prior to using such output.
Also one should realize that such output will not
contain lines ignored by the
-i ignore_file
process even if
-p 100
is used.
By default, 96% of the lines not ignored in host1 must
be unchanged.
That is, by default,
-p 90
is assumed.
-q hostid
By default, all newsgroup errors are reported on standard errors.
This flag quiets errors from
host1
or
host2 .
The
hostid
value is interpreted the same as in
-b .
-s size
If
size >0,
then ignore newsgroups with names longer than
size ,
and ignore newsgroups equivalenced to names longer than
size .
Length checking is perform on both the local and remote hosts.
By default,
size
is 0 and thus no length checking is performed.
-S spool_dir
For each new newsgroup (i.e., selected groups found on host2 that
were not found on host), the newsgroup directory under
spool_dir
will be examined.
If storageapi is turned on, this should be the same name as pathoverview in
inn.conf .
If this newsgroup directory exists, then the
hi & low (2nd & 3rd active fields)
values of the active entry will be changed to
reflect the range articles found.
This flag is only useful with
-o a,
-o a1,
-o ak,
-o aK,
-o alk,
-o alK,
-o ak1 or
-o aK1.
The
-S spool_dir
will override any hi & low (2nd & 3rd active fields)
values that would normally have been used.
This is an important and very much recommended
option as it will prevent article number collisions
on newsgroups that have been removed previous but still
have unexpired articles in them.
-t hostid
Ignore improper newsgroups with only a top component
from
host1
or
host2 .
The
hostid
value is interpreted the same as in
-b .
The following newsgroups are considered proper newsgroups
for top only names:
+0.5i
control
general
junk
test
to
-0.5i
For example, the following newsgroup names are improper because they
only contain a top level component:
+0.5i
dole_for_pres
dos
microsoft
windoes95
-0.5i
By default, all improper top level only newsgroups from the remote (
-t 2
) are ignored.
-T
This flag causes
host2
newsgroups from new hierarchies to be ignored.
Normally if only
host2
has the newsgroup
chongo.was.here
then it will be created for
host1 .
However if
host1
does not have any 'chongo.*' newsgroups and this
flag is given, then
chongo.was.here
will be ignored and will not be created on
host1 .
-v verbose_lvl
No default,
actsync (8)
is not verbose.
This flag controls the verbosity level as follows:
+0.5i
0 no debug or status reports (default)
1 print summary,
if work was needed or done
2 print actions, exec output & summary,
if work was needed or done
3 print actions, exec output & summary
4 full debug output
-z sec
If
-o x
is selected,
actsync (8)
will pause for
sec
seconds before each command is executed.
This helps prevent
innd (8)
from being busied-out if a large number of
ctlinnd (8)
commands are needed.
One can disable this sleeping by using
-z 0 .
By default,
actsync (8)
will pause for
4
seconds before each command is executed if
-o x
is selected.
-0.5i
EXAMPLES
Determine the difference (but don't change anything) between your
newsgroup set and uunet's set:
+0.5i
actsync news.uu.net
-0.5i
Same as above, with full debug and progress reports:
+0.5i
actsync -v 4 news.uu.net
-0.5o
Force a site to have the same newsgroups some other site:
+0.5i
actsync -o x master
-0.5i
This may be useful to sync a slave site to its master, or
to sync internal site to a gateway.
Compare your site with uunet, disregarding local groups and
certain local differences with uunet.
Produce a report if
any differences were encountered:
+0.5i
actsync -v 2 -i actsync.ign news.uu.net
-0.5i
where
actsync.ign
contains:
+0.5i
# Don't compare to.* groups as they will differ.
#
i to.*
# These are our local groups that nobody else
# (should) carry. So ignore them for the sake
# of the compare.
#
i nsa.*
# These groups are local favorites, so keep them
# even if uunet does not carry them.
#
i ca.dump.bob.dorman
i ca.keep.bob.dorman
i alt.tv.dinosaurs.barney.die.die.die
i alt.tv.dinosaurs.barney.love.love.love
i alt.sounds.* =alt.binaries.sounds.*
-0.5i
To interactively sync against news.uu.net, using the same
ignore file:
+0.5i
actsync -o xi -v 2 -i actsync.ign news.uu.net
-0.5i
Based on newsgroups that you decided to keep, one could
make changes to the
actsync.ign
file:
+0.5i
# Don't compare to.* groups as they will differ.
#
i to.*
# These are our local groups that nobody else
# (should) carry. So ignore them for the sake
# of the compare.
#
i nsa.*
# These groups are local favorites, so keep them
# even if uunet does not carry them.
#
i ca.dump.bob.dorman
i alt.tv.dinosaurs.barney.die.die.die
i alt.sounds.* =alt.binaries.sounds.*
# Don't sync test groups, except for ones that are
# moderated or that are under the gnu hierarchy.
i *.test
c *.test m # check moderated test groups
c gnu.*.test
c gnu.test # just in case it ever exists
-0.5i
Automatic processing may be setup by using the following
actsync.cfg
file:
+0.5i
# host to sync off of (host2)
host=news.uu.net
# location of the ignore file
ignore_file=<PREFIX specified with --prefix at configure>/etc/actsync.ign
# where news articles are kept
spool=<patharticles in inn.conf>
# actsync(8) flags
#
# Automatic execs, report if something was done,
# otherwise don't say anything, don't report
# uunet active file problems, just ignore
# the effect entries.
flags=-o x -v 2 -q 2
-0.5i
and then by running actsyncd with the path to the config
file.
+0.5i
actsyncd <PREFIX specified with --prefix at configure>/etc/actsync.cfg
-0.5i
One may produce a trial
actsyncd (8)
run without changing anything
on the server by supplying the debug_level arg:
+0.5i
actsyncd <PREFIX specified with --prefix at configure>/etc/actsync.cfg 2
-0.5i
The debug_level causes
actsyncd (8)
to run
actsync (8)
with an -v debug_level (overriding any -v
flag on the flags line),
prevents any changes from being made to the
active (5)
file, writes a new active file to standard output
and writes debug messages to standard error.
If the debug_outfmt arg is also given to
actsyncd (8)
then the data written to standard output will
be in -o debug_outfmt instead of in -o a1 format.
The following /bin/sh command:
+0.5i
actsyncd <PREFIX specified with --prefix at configure>/etc/actsync.cfg 4 >cmd 2>dbg
-0.5i
Will operate in debug mode,
not change the
active (5)
file, write
ctlinnd (8)
style commands to cmd and
write debug statements to dbg.
To check only the major hierarchies against news.uu,net, use the following
actsync.ign
file:
+0.5i
# by default, ignore everything
i *
# check the major groups
c comp.*
c gnu.*
c sci.*
c alt.*
c misc.*
c news.*
c rec.*
c soc.*
c talk.*
-0.5i
and running:
+0.5i
actsync -i actsync.ign news.uu.net
-0.5i
To determine the differences between your old active and
your current default server:
+0.5i
actsync <pathetc in inn.conf>/active.old -
-0.5i
To report but not fix any newsgroup problems with the current active file:
+0.5i
actsync - -
-0.5i
To detect any newsgroup errors on your local server, and
to remove any
*.bork.bork.bork
style silly newsgroup names:
+0.5i
actsync -b 2 - -
-0.5i
The active file produced by:
+0.5i
actsync ... flags ... -o x erehwon.honey.edu
-0.5i
or by:
+0.5i
actsync ... flags ... -o c erehwon.honey.edu | sh
-0.5i
is effectively the same as the active file produced by:
+0.5i
ctlinnd pause 'running actsync'
rm -f active.new
actsync ... flags ... -o a1 erehwon.honey.edu > active.new
rm -f active.old
ln active active.old
mv active.new active
ctlinnd reload active 'running actsync'
ctlinnd go 'running actsync'
-0.5i
It should be noted that the above 'pause', 'actsync', 'reload' and 'go'
method is faster. However, in order to avoid article number collisions
on newsgroups that have been removed previous but still
have unexpired articles in them, it is very much recommended
that the
-S spool_dir
be used with any of the
-o a*
flags.
Thus, a much better and safer version of the above would be:
+0.5i
ctlinnd pause 'running actsync'
rm -f active.new
actsync ... flags ... -o a1 -S <patharticles or pathoverview in inn.conf> erehwon.honey.edu > active.new
rm -f active.old
ln active active.old
mv active.new active
ctlinnd reload active 'running actsync'
ctlinnd go 'running actsync'
-0.5i
The above process is similar to what
actsyncd (8)
does by default.
CAUTION
Careless use of this tool may result in the addition
change or removal of newsgroups that you don't want.
You should avoid using the x output format until
you are sure it will do what you want.
Be aware that
innd (8)
servers older than version 1.5 may corrupt the active file when
multiple rmgroups are performed if the server is paused or throttled.
This is not a
actsync (8)
bug, it is a server bug.
Using the pause, actsync, reload and go method
noted above avoids this problem of older servers.
BUGS
If a newsgroup appears multiple times,
actsync (8)
will treat all copies as errors.
However, if the group is marked for removal, only
one rmgroup will be issued.
The timeout for
ctlinnd (8)
commands is fixed at 30 seconds when
running in ``x'' or ``xi'' output format.
Perhaps the timeout value should be controlled via a command line option?
SEE ALSO
HISTORY
Written by Landon Curt Noll <chongo@toad.com> for InterNetNews.
Updated to support ftp fetching by David Lawrence <tale@isc.org>.
|