Re: [foxboro] FW: system management can't communicate with the current CLAN initiator

Hani,
Sorry, I see my previous reply did not get through.

Have you resolved your LAN problem or still having faults?

Yes, your 700 pkts/sec nodes will cause problems such as you describe.  The
problem with heavily overloading a CBLAN is that once messages start getting
dropped due to high traffic, the system starts generating more traffic
trying to troubleshoot the problem.

You may also be having some problems with the FONBEs in your CCR node.  Is
this node 2 segments or 3?  Replacing the FONBEs with NCNI modules
(relatively inexpensive upgrade) should eliminate this as a possibility.
Nodes segmented with FONBEs have been prone to trouble at several sites I've
worked with.  High traffic on these nodes is more likely to cause islanding
of some stations.
You need to determine what is causing the high traffic to/from the two nodes
that are reaching 700 pkts/sec and correct.  If you are collecting large
amounts of data from these nodes to an Aspentech or PI data collector those
are likely causes.  Consider purchasing additional collectors and installing
on the high traffic nodes - this way data can be collected on the same node
it is generated without sending across the CBLAN.  It sounds like your
change to PI which brought the traffic down to 400 pkts/sec is helping.

You should be able to work with Foxboro to collect data from each of your
nodes and analyze the traffic to see what is going where.  The snoop command
can collect packets from your nodebus, and you or Foxboro can use Wireshark
or SnifferPro to analyze.

For troubleshooting the CBLAN - especially noise and/or physical problems,
follow the steps in Chapter 8 of the Network Cable Systems manual, B0193UW -
it is fairly detailed and a good guide.

If your problems just started in August, and there were no significant
changes to the system around that time, some possibilities include:
* Traffic may have been gradually increasing with additions to PI,
peer-to-peer, operator graphics, etc... until it rose high enough that
packets start to be dropped during periods of slightly higher than normal
traffic, which causes additional traffic and can rapidly overwhelm the LAN.
* You may have some physical problems - loose connections, corrosion,
grounding problems, etc... causing noise or other faults, adding to an
already overloaded LAN
* Some change may have been made that had an unforseen impact to cross-node
traffic.

Do you have plans in place to upgrade your CBLAN to a Mesh network?  This
can be a relatively simple upgrade and will correct your LAN loading
problems by providing much higher bandwidth between nodes.  If you have
adequate fiber in place, the changeover can be done at one time, eliminating
the need to reboot any controllers, and with no more impact to the system
than you already have when the CBLAN overloads.

Regards,

Kevin FitzGerrell

On Tue, Jun 9, 2009 at 2:13 PM, Ahmari, Hani A <hani.ahmari@xxxxxxxxxx>wrote:

> Thank you Mr. Kevin for your reply
>
> Our architecture is 10 nodebus nodes which has an extended nodebus in CCR
> and they are connected through fiber optic by using FONBE in both sides. CCR
> nodes are connected to CLAN by using DCLIFT.
> Normally the traffic in all PIB's are within average of 200 PCKT except two
> node which reached up to 700 PACT which causing freezing the CCR consoles to
> maximum 20 min. and we have 4 times freezing issue.
> Each node has system monitor but now I can't initiate a cable test
> initiator or change the current initiator.
> I found the current initiator by using (glof -p TBMSTR0001) and I reboot
> the current initiator (both CLAN FT cards) in same time and as a result of
> that the current initiator transferred to another node but the standing
> alarms still coming in all nodes.
>
> the time stamp for PI server had been modified and this modification
> limited the high traffic to maximum 400 PACKT which was before modification
> reached up to 700 PACKT. We have Aspetech application but it is in another
> node which is normal.
>
> We don't have any history of high traffic and/or LAN problems it just
> started since August 2008.
>
> How I can look for what could have suddenly caused either high traffic or
> noise?
>
> Are the points 2 & 3 will solve the issue the standing repeating alarms
> (MTBM NF8024 -00021 Failed takeover attempt by tokenbus master) & (CFRDF
> NF8024 -00021 Failed takeover attempt by tokenbus master)?
>
> Regards
> Hani Ahmari
>
>
> -----Original Message-----
> From: foxboro-bounce@xxxxxxxxxxxxx [mailto:foxboro-bounce@xxxxxxxxxxxxx]
> On Behalf Of Kevin Fitzgerrell
> Sent: Monday, June 08, 2009 2:16 PM
> To: foxboro@xxxxxxxxxxxxx
> Subject: Re: [foxboro] system management can't communicate with the current
> CLAN initiator
>
> Hani,
> What is your architecture?
> How many nodes?
> Copper, fiber or both?  Passive hubs?  Active concentrators?  Copper/fiber
> converters?  Splitter/combiners?
> How much traffic on each node (PDUS30 counter on each cblan)?
> Do you have a system monitor on each node?
>
> Yes, it is likely that the problem you are seeing is a result of high
> traffic, possibly exacerbated by noise or fault condition.
>
> Some ways to deal with this situation:
> 1) determine who is the cable test initiator and do a cable test from a
> system monitor in that node
> 2) reboot all LAN modules (both sides together if FT) and see if you
> recover.
> 3) try completely shutting down either the A or B side of the network and
> run your LAN modules single - may be able to reduce NFD traffic by
> simplifying the network.
> 4) shut the LAN down completely and build it back up one node at a time -
> preferably from most to least important - until you hit the maximum
> sustainable traffic level, or locate a fault
>
> If traffic is high, try to reduce unnecessary traffic - if PI/Aspentech are
> collecting data across nodes, shutting down the data collection may allow
> you to bring the LAN up.
>
> Have you had a history of high traffic and/or LAN problems?  If you have
> had
> no problems previously, look for what could have suddenly caused either
> high
> traffic or noise.
>
> Regards,
>
> Kevin FitzGerrell
>
> On Mon, Jun 8, 2009 at 1:38 PM, Ahmari, Hani A <hani.ahmari@xxxxxxxxxx
> >wrote:
>
> > There are standing alarms coming in all nodes every 5 minutes states that
> > (MTBM NF8024 -00021 Failed takeover attempt by tokenbus master), and the
> > reason of these alarms is the high traffic which is generating from one
> > node. And also there is a problem with system management states that the
> >  system management can't communicate with the current CLAN initiator and
> due
> > of that; RUN CARREIER BAND CABLE TEST and CHANGE CARRIER BAND TEST
> INITIATOR
> > cannot be executed.
> > Is the problem of the system management is a result of high traffic?
> >
> > I need your support to resolve the issue of the system management can't
> > communicate with the current CLAN initiator?
> >
> > Regards
> > Hani
> >
> >
> > ________________________________
> > The contents of this email, including all related responses, files and
> > attachments transmitted with it (collectively referred to as "this
> Email"),
> > are intended solely for the use of the individual/entity to whom/which
> they
> > are addressed, and may contain confidential and/or legally privileged
> > information. This Email may not be disclosed or forwarded to anyone else
> > without authorization from the originator of this Email. If you have
> > received this Email in error, please notify the sender immediately and
> > delete all copies from your system. Please note that the views or
> opinions
> > presented in this Email are those of the author and may not necessarily
> > represent those of Saudi Aramco. The recipient should check this Email
> and
> > any attachments for the presence of any viruses. Saudi Aramco accepts no
> > liability for any damage caused by any virus/error transmitted by this
> > Email.
> >
> >
> >
> > _______________________________________________________________________
> > This mailing list is neither sponsored nor endorsed by Invensys Process
> > Systems (formerly The Foxboro Company). Use the info you obtain here at
> > your own risks. Read http://www.thecassandraproject.org/disclaimer.html
> >
> > foxboro mailing list:             http://www.freelists.org/list/foxboro
> > to subscribe:         mailto:foxboro-request@xxxxxxxxxxxxx?subject=join
> > to unsubscribe:      mailto:foxboro-request@xxxxxxxxxxxxx?subject=leave
> >
> >
>
>
>
>
> _______________________________________________________________________
> This mailing list is neither sponsored nor endorsed by Invensys Process
> Systems (formerly The Foxboro Company). Use the info you obtain here at
> your own risks. Read http://www.thecassandraproject.org/disclaimer.html
>
> foxboro mailing list:             http://www.freelists.org/list/foxboro
> to subscribe:         mailto:foxboro-request@xxxxxxxxxxxxx?subject=join
> to unsubscribe:      mailto:foxboro-request@xxxxxxxxxxxxx?subject=leave
>
>
> The contents of this email, including all related responses, files and
> attachments transmitted with it (collectively referred to as "this Email"),
> are intended solely for the use of the individual/entity to whom/which they
> are addressed, and may contain confidential and/or legally privileged
> information. This Email may not be disclosed or forwarded to anyone else
> without authorization from the originator of this Email. If you have
> received this Email in error, please notify the sender immediately and
> delete all copies from your system. Please note that the views or opinions
> presented in this Email are those of the author and may not necessarily
> represent those of Saudi Aramco. The recipient should check this Email and
> any attachments for the presence of any viruses. Saudi Aramco accepts no
> liability for any damage caused by any virus/error transmitted by this
> Email.
>


 
 
_______________________________________________________________________
This mailing list is neither sponsored nor endorsed by Invensys Process
Systems (formerly The Foxboro Company). Use the info you obtain here at
your own risks. Read http://www.thecassandraproject.org/disclaimer.html
 
foxboro mailing list:             http://www.freelists.org/list/foxboro
to subscribe:         mailto:foxboro-request@xxxxxxxxxxxxx?subject=join
to unsubscribe:      mailto:foxboro-request@xxxxxxxxxxxxx?subject=leave

Other related posts: