Re: [foxboro] Rebuilding CSA database

Kevin,

Thanks for the info.  I never had to do a CSA_Merge, or use any of those 
related tools, fortunately, but it is useful information only a browser 
away if ever excrement and turbine meet.


"I do appreciate the reason for the separate databases, but given the
frequency that people have problems with them not being in-synch, I think
that we should have tools as part of the main distribution to rebuild 
and/or
re-synchronize."


I don't.  Multiple databases introduce more potential points of failure, 
and there is not really full redundancy between the databases so having 
more than one is not much use in terms of recovery. I have worked on a 
system that didn't have separate databases for this info and it worked 
just fine.  Having said that, though, with graphical configurators 
becoming all the rage now, I think some separation of databases will be 
inevitable, as you generally don't want your block connection drawings 
downloaded into your controllers.  I just hope Invensys (and the other 
controls vendors we use) realize the potential problems with this and 
provide resync/repair tools as you stated.


Corey


P.S. Side note - below link is an article about an Australian power grid 
authority that was running Windows PCs as X-terminals/operator consoles on 
a Solaris-based SCADA system.  They had a virus hit the PCs, causing them 
to lose view of the SCADA servers.  The organization's admins (who 
apparently like Unix but weren't too adept with firewalls and network 
design) swapped the infected Windows PCs out for Linux machines they were 
testing and got the consoles back up.

http://www.theinquirer.net/inquirer/news/1556944/linux-saves-aussie-electricity





Kevin Fitzgerrell <fitzgerrell@xxxxxxxxx> 
Sent by: foxboro-bounce@xxxxxxxxxxxxx
10/08/2009 02:16 AM
Please respond to
foxboro@xxxxxxxxxxxxx


To
foxboro@xxxxxxxxxxxxx
cc

Subject
[foxboro] Rebuilding CSA database






Hi all,
Just in case anyone else runs in to this problem, my work this afternoon 
may
save someone else some time.

We noticed a massive problem with our CSA database this morning.  An
engineer was trying to copy a block while creating a new one and ICC was
saying "Find Object" Failed.  I ran a CSA_Save and checked the files
generated, many of our control station CSA files from the save were empty.
Others are not completely empty, but don't have all their blocks.  I
confirmed in /opt/fox/csa that the idx files were effectively empty too. I
suspect that somewhere during our move to the Mesh, or during a 
workstation
change-over, some engineer (could even be me) messed up a CSA_Merge.

After a conversation with TAC, we were advised that if
saveall/initialize/loadall is impractical (yes) to do a CSA_Save and then
manually edit in the missing or corrupted information in the same format.
An effective middle ground would be to build the CSA database up off-line 
on
a testbed by doing a loadall of each of the control stations.  Well, we 
have
around a hundred control stations, about half of which are affected, I
really didn't feel like manually entering 20,000 lines of CSA, and don't 
see
the need to build the database up off-line, so I wrote a little script to
create a CSA save for me based on iccprt output from the control stations.
We rarely use our work volumes, so I'm not recovering those, but they 
don't
appear corrupted or bad either, so I'm collecting them from the real CSA
save.

My script, build_csa_from_prt.sh is as follows:

#!/bin/sh
HOME="/opt/ts_group"
cd $HOME
mkdir save
PRT_FILES=`ls *.prt`
for FILE in $PRT_FILES
do
  OUT_FILE=`echo $FILE | sed 's/.prt//'`
  egrep '(^NAME|^  TYPE)' $FILE | sed -f merge_name_type.sed | sort -u | 
awk
'$3 ~/COMPND/{printf ("%-12s\n", $1)} $3 !~ /COMPND/{printf
("%-12s\t%-12s\t%-6s\n", $1, $2, $3)}' > $HOME/save/$OUT_FILE
done
echo "now copy volumes in from last CSA save on CSA server"
The "egrep" line is long and will probably wrap - starts with "egrep" and
finishes with "$OUT_FILE".  I have a current iccprt output from all of my
control stations in the directory I'm running this in, named 
STATIONLBUG.prt

There is a sed script used in there to take multi-line NAME and TYPE and
turn that in to a single line and edit appropriately.  The script
merge_name_type.sed looks like:
/^NAME/{
N
s/NAME   = //
s/:/ /
s/\n  TYPE   = / /
s/ COMPND/ # COMPND/
}
Additionally, you should check to see if there are any block name 
mismatches
between the control stations and the station work files.  There are a good
set of tools for checking mismatches as part of the OLUG toolkit, but I
wrote the following check_mm.sh script to compare the same STATIONLBUG.prt
files on a host to the checkpoint files on that same host for block
mismatches:
#!/bin/sh
HOST=`uname -n`
HOME="/opt/ts_group"
STATIONS=`grep $HOST /usr/fox/sp/sldb | egrep '(^CP|^GW|^AB)' | awk 
'{print
$1}'`
cd /usr/fox/sp/files
for STATION in $STATIONS
do
  /opt/fox/bin/tools/dbvu -p -CDB${STATION}.UC | egrep '(^ 200|:)' | egrep
-v '(200  CMP|^CHECKPOINT|^BPC)' | awk '{print $2}' > $HOME/mm_1.tmp
  grep "^NAME" $HOME/${STATION}.prt | awk '{print $3}' > $HOME/mm_2.tmp
  diff $HOME/mm_1.tmp $HOME/mm_2.tmp > $HOME/MM_${STATION}.out
done
cd $HOME
more $HOME/MM_*.out > $HOME/mismatch_report.$HOST

The "dbvu" line is long and will probably wrap.  It starts with
"/opt/fox/bin/tools/dbvu" and ends with "mm_1.tmp"

I do appreciate the reason for the separate databases, but given the
frequency that people have problems with them not being in-synch, I think
that we should have tools as part of the main distribution to rebuild 
and/or
re-synchronize.

Regards,

Kevin FitzGerrell





 
 
_______________________________________________________________________
This mailing list is neither sponsored nor endorsed by Invensys Process
Systems (formerly The Foxboro Company). Use the info you obtain here at
your own risks. Read http://www.thecassandraproject.org/disclaimer.html
 
foxboro mailing list:             http://www.freelists.org/list/foxboro
to subscribe:         mailto:foxboro-request@xxxxxxxxxxxxx?subject=join
to unsubscribe:      mailto:foxboro-request@xxxxxxxxxxxxx?subject=leave
 

Other related posts: