Update of /cvsroot/ssic-linux/openssi/docs/debian
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16109/debian
Added Files:
README.ntp README.upgrade
Log Message:
Reorganization of openssi/docs to be common across all distro branches
--- NEW FILE: README.ntp ---
This versions was originally written for RedHat, but has had some
changes added for Debian. Volunteers are welcome to fully "Debian-ize"
it.
The README assumes that ntp was configured on the original
installation node and that ssi has been installed.
NOTE: I am trying these things out as I write this, for the most
part, but your mileage may vary.
NOTE: Once you perform the following procedures, the Red Hat
"dateconfig" GUI cannot be used to change the ntp settings.
NOTE: All shell commands are for bash.
First, make sure ntp is off.
/sbin/service ntpd off
Second, we may need to use ntp authentication, so generate the ntp
keys file.
/usr/sbin/ntp-genkeys -f
Third, each node will need an individual "drift" file. Edit your
ntp.conf and make sure there is a driftfile line like the following:
driftfile /var/lib/ntp/drift
If the directory /var/lib/ntp already exists:
mkdir -p /cluster/node\{nodenum\}/var/lib
mv /var/lib/ntp /cluster/node\{nodenum\}/var/lib
Make /cluster/node<node>/var/lib/ directories for all the nodes in the cluster.
Note for Debian users, there may be no ntp user or group.
( cd /cluster; for d in node*; do
mkdir -p $d/var/lib/ntp; chown ntp:ntp $d/var/lib/ntp;
chmod 0755 $d/var/lib/ntp; done )
Make a context-symlink in /var/lib for the ntp directory:
ln -s /cluster/node\{nodenum\}/var/lib/ntp /var/lib/ntp
*** Configuration #1: All nodes have access to an external timeserver
This is a very straightforward configuration. All nodes use the same
configuration files with the exception of the drift file we have
already set up.
Edit your /etc/ntp.conf file:
Assuming your ICS network is 192.168.0.0/255.255.255.0, add the
following lines to /cluster/etc/ntp.conf (The key can be any
one of the keys generated in /etc/ntp/keys by ntp-genkeys.):
restrict 192.168.0.0 mask 255.255.255.0
broadcast 192.168.0.255 key 1
broadcastclient
trustedkey 1
requestkey 1
This allows all the machines on the ICS network for to use the
machine as a broadcast server and all nodes in the cluster to
use the other's idea of the time. The "requestkey" line is
needed so a script can modify the server configuration on the fly.
If the following lines are present in the ntp.conf, they should be
deleted or commented out:
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
This lines allow ntpd to use the local clock as a time source
that ntp clients can synchronize to. We want only one node in
the cluster to have this setting at any given time: the
current CLMS master. A script named ssi-ntpsetrefclk will
be run from /etc/rc.d/rc and /etc/rc.d/rc.sysrecover to set
the local clock as a reference clock.
Add the ICS addresses for all the nodes to
/cluster/etc/ntp/step-tickers. (Any external servers should already be there.)
Start ntp
/sbin/service ntpd start
Assuming your networking is setup correctly, you should see a series
of messages similar to the following:
(node 1)
ntpd: Synchronizing with time server: [ OK ]
Starting ntpd: [ OK ]
(node 2)
ntpd: Synchronizing with time server: [ OK ]
Starting ntpd: [ OK ]
*** Configuration #2: A limited number of nodes have access to the external
time server. The nodes with the external access should synchronize to
the external clocks; all other nodes should synchronize with those
nodes via the ICS network.
For the nodes in communication with the external network, you need one
ntp.conf file; for the nodes that are only on the internal network you
need another. This means we need another set of context-symlinks.
mv /etc/ntp.conf /cluster/etc
ln -s /cluster/node\{nodenum\}/etc/ntp.conf /etc/ntp.conf
Set up /cluster/etc/ntp.conf as for #1. This ntp.conf file is used only by
the nodes with access to the external time servers. (Note: all possible CLMS
masters must have external access for things to work properly.) For the
step-tickers file, add the ICS addresses of only those nodes that can talk to
the external servers. Note Debian does not use step-tickers. See below.
Copy the /cluster/etc/ntp.conf to /cluster/etc/ntp.conf.dep and remove
the lines referring to external servers and the "broadcast" line and
the "requestkey" line. (The "broadcastclient" line is still required.)
This ntp.conf will be used by nodes without external access. (These
nodes are ntp clients only.)
For each node that has access to an external time server:
ln -s /cluster/etc/ntp.conf /cluster/node<number>/etc/ntp.conf
For example node 1:
ln -s /cluster/etc/ntp.conf /cluster/node1/etc/ntp.conf
For all nodes that do not have access to the external time servers:
ln -s /cluster/etc/ntp.conf.dep /cluster/node<number>/etc/ntp.conf
Also, assuming that any node now added to the cluster will not have
access to the external network, make the following symlink as well.
ln -s /cluster/etc/ntp.conf.dep /cluster/nodetemplate/etc/ntp.conf
All new nodes will get this symlink when they are added.
RedHat passes the servers in the step-tickers file to the ntpdate
command and uses it to set the time on boot. The ntp-server on the
CLMS master will take approximately 1 minute to stabilize. During this
window, nodes joining the cluster will get the following error:
ntpd: Synchronizing with time server: [FAILED]
The time on the node should have been set within a second by via the
ssi-timesync script and ntpd should begin to correct the time once the
server on the CLMS master node becomes ready.
For Debian, there is no step-tickers file. Make sure the ntpdate package is
installed, and update the /etc/default/ntp-servers with your favorite
timeserver.
Add "ntpdate initnode Y" to /etc/rc.nodeinfo.
##################################################
### begin Debian ntp.conf for Configuration #2 ###
##################################################
# /etc/ntp.conf, configuration for ntpd
broadcast 192.168.0.255 ttl 6
restrict 192.168.0.0 mask 255.255.255.0
# ntpd will use syslog() if logfile is not defined
logfile /var/log/ntpd
driftfile /var/lib/ntp/ntp.drift
#statsdir /var/log/ntpstats/
#statistics loopstats peerstats clockstats
#filegen loopstats file loopstats type day enable
#filegen peerstats file peerstats type day enable
#filegen clockstats file clockstats type day enable
### lines starting 'server' are auto generated,
### use dpkg-reconfigure to modify those lines.
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
# Need a few external sources before your own server
# believes in its own time serving capabilities
server time_server_01
server time_server_02
server time_server_03
authenticate yes
keys /var/lib/ntp/ntp.keys
################################################
### end Debian ntp.conf for Configuration #2 ###
################################################
######################################################
### begin Debian ntp.conf.dep for Configuration #2 ###
######################################################
# /etc/ntp.conf, configuration for ntpd
broadcastclient
# ntpd will use syslog() if logfile is not defined
logfile /var/log/ntpd
driftfile /var/lib/ntp/ntp.drift
#statsdir /var/log/ntpstats/
#statistics loopstats peerstats clockstats
#filegen loopstats file loopstats type day enable
#filegen peerstats file peerstats type day enable
#filegen clockstats file clockstats type day enable
### lines starting 'server' are auto generated,
### use dpkg-reconfigure to modify those lines.
#server 127.127.1.0 # local clock
#fudge 127.127.1.0 stratum 10
authenticate yes
keys /var/lib/ntp/ntp.keys
####################################################
### end Debian ntp.conf.dep for Configuration #2 ###
####################################################
Wait about five minutes, and check your initnode with ntpq:
# ntpq
ntpq> pe
remote refid st t when poll reach delay offset jitter
==============================================================================
192.168.0.255 0.0.0.0 16 - - 64 0 0.000 0.000 4000.00
LOCAL(0) LOCAL(0) 10 l 39 64 377 0.000 0.000 0.008
+cudns.cit.corne gps1.tns.its.ps 2 u 98 256 377 46.723 -7.012 5.960
+louie.udel.edu huey.udel.edu 2 u 27 256 377 23.131 -1.695 0.874
*ns1.usg.edu ntp0.mcs.anl.go 2 u 229 256 377 35.252 -4.181 3.286
ntpq> as
ind assID status conf reach auth condition last_event cnt
===========================================================
1 364 8000 yes yes none reject
2 365 9014 yes yes none reject reachable 1
3 366 9414 yes yes none candidat reachable 1
4 367 9414 yes yes none candidat reachable 1
5 368 9614 yes yes none sys.peer reachable 1
ntpq> rv
status=0664 leap_none, sync_ntp, 6 events, event_peer/strat_chg,
version="ntpd 4.1.2a@1:4.1.2a-2 Tue Nov 11 11:33:28 UTC 2003 (2)",
processor="i686", system="Linux2.4.20-pre3", leap=00, stratum=3,
precision=-17, rootdelay=52.037, rootdispersion=29.729, peer=368,
refid=ns1.usg.edu,
reftime=c3df82e7.4c49cb25 Thu, Feb 19 2004 13:45:59.298, poll=8,
clock=c3df83d4.c7574f72 Thu, Feb 19 2004 13:49:56.778, state=4,
offset=-2.768, frequency=-185.529, jitter=4.524, stability=0.030
ntpq>
*** Configuration #3: A cluster without access to any external ntp
timeservers. All nodes should synchronize with the CLMS master via the
ICS network.
As #2 above, except there are no external servers listed in the
ntp.conf or step-tickers files. The ntp.conf should be used by all the
potential master nodes in the cluster; the step-tickers file should
contain all the potential master nodes in ther cluster. The
ntp.conf.dep should be used by all non-master nodes.
The ntp.conf I used follows.
# Prohibit general access to this service.
restrict default ignore
# External servers
restrict 192.168.0.10 mask 255.255.255.255 nomodify notrap noquery
server 192.168.0.10 # linuxbuild1
restrict 192.168.0.11 mask 255.255.255.255 nomodify notrap noquery
server 192.168.0.11 # kahuna
restrict 192.168.0.0 mask 255.255.255.0
broadcast 192.168.0.255 key 1
broadcastclient
trustedkey 1
requestkey 1
# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
# -- CLIENT NETWORK -------
# Permit systems on this network to synchronize with this
# time service. Do not permit those systems to modify the
# configuration of this service. Also, do not use those
# systems as peers for synchronization.
# restrict 192.168.1.0 mask 255.255.255.0 notrust nomodify notrap
# --- OUR TIMESERVERS -----
# or remove the default restrict line
# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
# restrict mytrustedtimeserverip mask 255.255.255.255 nomodify notrap noquery
# server mytrustedtimeserverip
# --- NTP MULTICASTCLIENT ---
#multicastclient # listen on default 224.0.1.1
# restrict 224.0.1.1 mask 255.255.255.255 notrust nomodify notrap
# restrict 192.168.1.0 mask 255.255.255.0 notrust nomodify notrap
# --- GENERAL CONFIGURATION ---
#
# Undisciplined Local Clock. This is a fake driver intended for backup
# and when no outside source of synchronized time is available. The
# default stratum is usually 3, but in this case we elect to use stratum
# 0. Since the server line does not have the prefer keyword, this driver
# is never used for synchronization, unless no other other
# synchronization source is available. In case the local host is
# controlled by some external source, such as an external oscillator or
# another protocol, the prefer keyword would cause the local host to
# disregard all other synchronization sources, unless the kernel
# modifications are in use and declare an unsynchronized condition.
#
#server 127.127.1.0 # local clock
#fudge 127.127.1.0 stratum 10
#
# Drift file. Put this in a directory which the daemon can write to.
# No symbolic links allowed, either, since the daemon updates the file
# by creating a temporary in the same directory and then rename()'ing
# it to the file.
#
driftfile /var/lib/ntp/drift
broadcastdelay 0.008
#
# Authentication delay. If you use, or plan to use someday, the
# authentication facility you should make the programs in the auth_stuff
# directory and figure out what this number should be on your machine.
#
authenticate yes
#
# Keys file. If you want to diddle your server at run time, make a
# keys file (mode 600 for sure) and define the key number to be
# used for making requests.
#
# PLEASE DO NOT USE THE DEFAULT VALUES HERE. Pick your own, or remote
# systems might be able to reset your clock at will. Note also that
# ntpd is started with a -A flag, disabling authentication, that
# will have to be removed as well.
#
keys /etc/ntp/keys
--- NEW FILE: README.upgrade ---
Upgrading an OpenSSI Cluster from previous releases on
Debian 3.1(Sarge)
These instructions describe how to upgrade an OpenSSI
cluster on Debian Testing (3.1, Sarge) from previous
releases of openssi. Openssi can be upgraded on Debian
using apt-get utility following the steps mentioned below.
1. Comment out the entry used for installing 'openssi
1.0.0' in the file "/etc/apt/sources.list" and add
following entries in addition to entries used for
Debian installation.
deb http://www.openssi.org/contrib/debian/openssidebs-devel ./
deb-src http://www.openssi.org/contrib/debian/openssidebs-devel ./
2. Add following entries to /etc/apt/preferences if it
is not already there.
Package: *
Pin: origin www.openssi.org
Pin-Priority: 1001
3. Configure http proxy. In the bash shell , you can
export environment variable "http_proxy" by setting its
value to local proxy server.
4. Add necessary drivers list to
"/etc/mkinitrd/modules". Most important drivers are
network drivers, depending upon the network cards
used in the participating nodes in the cluster (Ex:
e100, eepro100 etc.), which would be used while
booting cluster nodes. If it is already there , no problem.
5. Execute:
# apt-get update
# apt-get dist-upgrade
This would upgrade openssi from previous release to
new release . Please see the 'known problems' in this
document if apt-get fails due to any error or configuration
and get the work around with few manual steps as
mentiond.
6. Reboot the cluster .
NOTE: It has been observed that some time tftp server
will not respond to request once it already responded
to client's request. So restart the inetd on the init
node if client could get IP address , but could not
continue booting.
# invoke-rc.d inetd restart
7. After booting execute(If few manual steps are
executed as mentioned in the workaround for the kown problems).
# apt-get dist-upgrade.
Known Problems
1. Initrd doesn't get unmounted.
2. Added new configuration "OPENSSI_CLUSTER=yes" in the
/etc/mkinitrd.conf. Either allow the mkinitrd.conf to
be rewritten while upgrading the mkinitrd package Or
add above entry manually and rebuild the ramdisk. You
can follow the mkinitrd manpage to find out how to
rebuild the ramdisk. It is shown below how to use
mkinitrd.
3. If you are creating the kernel and initrd image
manually, make sure the kernel name and initrd name
should follow the format shown below.. The file
/cluster/kernel_version is written when installing
the kernel package.
VERSION=`cat /cluster/kernel_version`
kernel="/boot/vmlinuz-"$VERSION
initrd="/boot/initrd.img-"$VERSION
Ex:
/usr/sbin/mkinitrd -o
/boot/initrd.img-2.4.22-ac1-ssi-686-smp
2.4.22-ac1-ssi-686-smp
4. 'apt-get dist-upgrade' may fail with following error
messages while upgrading from openssi 1.0. This is
due to a bug with the OpenSSI 1.0 release kernel . To
fix this run the mkinitrd command and create ramdisk
image as shown below. Please see man page of mkinitrd
if you need more details about mkinitrd and how to
use it.
# /usr/sbin/mkinitrd -o
/boot/initrd.img-2.4.22-ac1-ssi-686-smp 2.4.22-ac1-ssi-686-smp
Error Messages:
dpkg: dependency problems prevent configuration of
kernel-image-2.4-686-smp: kernel-image-2.4-686-smp
depends on kernel-image-2.4.22-ac1-ssi-686-smp;
however: Package
kernel-image-2.4.22-ac1-ssi-686-smp is not
configured yet. dpkg: error processing
kernel-image-2.4-686-smp (--configure): dependency
problems - leaving unconfigured dpkg: dependency
problems prevent configuration of openssi: openssi
depends on kernel-image-2.4-686-smp; however:
Package kernel-image-2.4-686-smp is not configured
yet. dpkg: error processing openssi (--configure):
dependency problems - leaving unconfigured Errors
were encountered while processing:
kernel-image-2.4.22-ac1-ssi-686-smp
kernel-image-2.4-686-smp openssi E: Sub-process
/usr/bin/dpkg
returned an error code (1)
5. Update the grub configuration, if the new kernle is
not listed in the "menu.list"(or any grub configuration
file that you are following). The below command will
add the new kernel and initrd to the menu.lst file .
# /sbin/update-grub
6. While creating initrd images using mkinitrd during
upgrade, you may see following messages. These
messages are expected and would not cause any problem
in upgrading opessi.
cpio: ./dev/cciss: No such file or directory cpio:
./dev/ide: No such file or directory cpio:
./dev/mapper: No such file or directory cpio:
./dev/md: No such file or directory cpio: ./dev/scsi:
No such file or directory
Maintained by Gopalakrishna <gopalak@xxxxxx>
-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
|
