Thanks for getting back to me Terry. I recall reading about that CAD problem vs the new workstation a few years ago - nasty! Last year we added a couple Windoz I/A V8.5 boxes with some mesh and a CP270 and ATS into our predominantly Unix I/A V6.5 & 7.1 environment and everything seemed fine for several months. Since getting blindsided by the CAD crash, we've learned that the highest version I/A box is supposed to be the DEV_MONITOR master at all times; in our case the Windoz AW. All indications say is was, but apparently somebody took exception to that and they fought about it at length. We have 9 AIM* packages running on 9 separate AWs and all CADs seized up solid about the same time. I'm sure all users could breathe easy if they knew their I/A system could/would somehow alarm or draw attention to the fact that CAD is broke. Here's hoping someone has and will share that silver bullet ... Jason Slade Control Technologist Protection & Control Systems Support Nanticoke G.S. -----Original Message----- From: foxboro-bounce@xxxxxxxxxxxxx [mailto:foxboro-bounce@xxxxxxxxxxxxx] On Behalf Of Terry Doucet Sent: January 18, 2012 2:11 PM To: foxboro@xxxxxxxxxxxxx Subject: Re: [foxboro] CAD Failure Jason,The closest problem for me to what you describe occurred when someone added new WP to the system without telling the rest of the system about this WP. At some later time, this WP took over as the Device Monitor MASTER for the system. Since this Device Monitor Master could not detect AW's, WP's and printers, it instructed the CP's to stop sending alarms to those devices. I cannot remember if the Historian stopped receiving alarms or not but no CAD's received any alarms. Of course, you have to realize that no alarms are coming before you know that the CAD is dead. Device Monitor was working correctly it was the installer who goofed. Is there one AW or WP (capable of being Device Monitor Master) that is off by itself on a section of your network? Since the Historian kept receiving alarms perhaps the problem is with the AW running historian. Terry > Subject: [foxboro] CAD Failure > Date: Wed, 18 Jan 2012 12:46:45 -0500 > From: jason.slade@xxxxxxx > To: foxboro@xxxxxxxxxxxxx > > We recently had a plant-wide CAD failure with all 46 workstations > being affected. Due to the plant's running mode at the time, it took > OPS a while to notice that there were no process alarms coming from > dozens of CPs to CAD. The historian however was still receiving the > messages!? > No SYS_MON alarm - nothing to bring this failure to the attention of > OPS. Even with all the log files, TAC could not ascertain what was > actually happening or what the cause was. One particular log file > indicated that there was a fight going on for control of DEV_MONITOR. > > We got CAD back up and running, but I was wondering: > - has anyone else been hit by this problem? > - does anyone have a site implemented work-around to bring to OPS > attention that CAD/DEV_MONITOR has failed? > > Thanks, > > Jason Slade > > Control Technologist > Protection & Control Systems Support > Nanticoke GS > > ----------------------------------------- > THIS MESSAGE IS ONLY INTENDED FOR THE USE OF THE INTENDED > RECIPIENT(S) AND MAY CONTAIN INFORMATION THAT IS PRIVILEGED, > PROPRIETARY AND/OR CONFIDENTIAL. If you are not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying, conversion to hard copy or other > use of this communication is strictly prohibited. If you are not the > intended recipient and have received this message in error, please > notify me by return e-mail and delete this message from your system. > Ontario Power Generation Inc. > > > ______________________________________________________________________ > _ This mailing list is neither sponsored nor endorsed by Invensys > Process Systems (formerly The Foxboro Company). Use the info you > obtain here at your own risks. Read > http://www.thecassandraproject.org/disclaimer.html > > foxboro mailing list: //www.freelists.org/list/foxboro > to subscribe: mailto:foxboro-request@xxxxxxxxxxxxx?subject=join > to unsubscribe: mailto:foxboro-request@xxxxxxxxxxxxx?subject=leave > _______________________________________________________________________ This mailing list is neither sponsored nor endorsed by Invensys Process Systems (formerly The Foxboro Company). Use the info you obtain here at your own risks. Read http://www.thecassandraproject.org/disclaimer.html foxboro mailing list: //www.freelists.org/list/foxboro to subscribe: mailto:foxboro-request@xxxxxxxxxxxxx?subject=join to unsubscribe: mailto:foxboro-request@xxxxxxxxxxxxx?subject=leave _______________________________________________________________________ This mailing list is neither sponsored nor endorsed by Invensys Process Systems (formerly The Foxboro Company). Use the info you obtain here at your own risks. Read http://www.thecassandraproject.org/disclaimer.html foxboro mailing list: //www.freelists.org/list/foxboro to subscribe: mailto:foxboro-request@xxxxxxxxxxxxx?subject=join to unsubscribe: mailto:foxboro-request@xxxxxxxxxxxxx?subject=leave