Re: Firewall Service dies like clockwork

  • From: Phill Hardstaff <phillh@xxxxxxx>
  • To: "'[ISAserver.org Discussion List]'" <isalist@xxxxxxxxxxxxx>
  • Date: Sun, 1 Dec 2002 14:51:54 +1100

Hi all, I can definitely confirm that I have EXACTLY the same issue as
described below in the article quoted by Tom, 
 
IF you have UDP based publishing (any UPD not just DNS) AND a Site and
Content filter containing FQDN the following happens;

and the firewall encounters huge amounts of UDP traffic (in my case DNS)
your firewall service will loose the plot and stop passing traffic, bug ?,
what it does mean though is I cannot use a feature of ISA that I paid for in
good faith, i.e. Site and Content Filters, my only solution was to drop the
three I had, no big deal, but ..
 
Anyone at MS who might be interested I can make this happen anytime I like
:) so it is reproducible at least, I could also document the exact setup and
what is happening.
 
Having said that, anyone really interested in doing filtering should check
out http://www.contentkeeper.com <http://www.contentkeeper.com>  as this is
most likely what I will be doing instead of trying to keep my own lists up
to date, this product rocks.
 
Cheers
 
Phill

-----Original Message-----
From: Thomas W Shinder [mailto:tshinder@xxxxxxxxxxxxxxxxxx]
Sent: Tuesday, 26 November 2002 4:29 PM
To: [ISAserver.org Discussion List]
Subject: [isalist] Re: Firewall Service dies like clockwork


http://www.ISAserver.org

http://www.ISAserver.org


Hey guys,
 
Here's some information I have that might be useful in this situation:
 

Junior Member 
Member # 8800 

          <http://forums.isaserver.org/ubb/icons/icon1.gif> posted November
15, 2002 12:13 PM
<http://forums.isaserver.org/ultimatebb.cgi?ubb=get_profile;u=00008800>
Profile for Jorgen
<http://forums.isaserver.org/ultimatebb.cgi?ubb=edit_post;f=6;t=001219;reply
_num=000010;u=00008800> Edit/Delete Post
<http://forums.isaserver.org/ultimatebb.cgi?ubb=reply;f=6;t=001219;replyto=0
00010> Reply With Quote  

  _____  

** NEWS FLASH **
Microsoft has managed to reproduce our problem in their lab AND they have
managed to find the cause.. no cure yet though :-(

IF you have UDP based publishing (any UPD not just DNS) AND a Site and
Content filter containing FQDN the following happens;

Incoming UDP requests are checked against the Site and Content rules by
attempting to do a reverse lookup on the incoming IP (to find a FQDN to
match against the IP).

If this for some reason fails (like the requesting IP not being in a reverse
zone) then the ISA tries to make a NBTSTAT query against the remote IP to
find the FQDN. 

Once it has succeded, failed or timed-out on the incoming request it will
then process the request.

This can take some time (at least on my side we just drop incoming netbios
so those will have to timeout) and during that time the ISA is gobbling up
UDP connections.

With heavy traffic this will at times cause the pool of available UDP
mapppings to be full so that incoming requests first have to wait for
another request to make it through the S&S rules before itself can start the
path through!

So if;
- you have a remote client that is not in a reverse zone and that can not be
resolved by nbtstat AND if it is re-requesting the DNS information after say
5 seconds 
- then you can easily end up in a situation where 
- the requests are being held pending, in wait for a process slot, while the
ISA is trying to resolve the FQDN of a previous request FROM THE SAME
MACHINE!

So the good news is that they know why and the bad news is that it sounds
like it is a fundamental change that needs to be done!

Possible workaround, no S&S rules! Not sure I want to go that way....

Was this clear? If not, drop me a line and I'll try again....

/Jörgen 
 
 
HTH,
Tom
Thomas W Shinder

[Phill Hardstaff] 
 I posted something about 3 months back about how my firewall service
stopped working at 0200h every Sunday morning and would not come back up
until 0430h with a reboot or restart the Firewall service. Anyway, after
months of pulling my hair out I finally nailed it, and looking back over
some posts on the list about problems with the firewall service I believe
others are probably having this problem as well.

 
What happens : 0200h Sunday morning, firewall service dies, web proxy still
runs OK, can still access web site by IP, published servers stop working
etc.
 
Last Friday I ran a Webtrends report on my web server, which is behind the
firewall, this is also my DNS server, this was during the day, within 10
minutes I had these same symptoms that I outlined above, I stopped and
restarted the firewall, I rebooted it but it would just die straight away. I
had hadn't made the link with Webtrends yet, but I shutdown the web server
and while it was off found I could stop and start the firewall service and
IT WORKED. I bought the web server back up and had a look at Webtrends and
found I had all my reports set to run at 0200h Sunday morning (they run
sequentially), boy was I happy. I disabled all the Webtrends reports and
shut down the scheduler. By this time I had to leave work to catch a plane,
I checked on Sunday morning and for the first time in 3-4 months the
firewall service didn't stop at 0200h on Sunday morning.
 
Why ? Webtrends does DNS lookups while running to resolve IP's from the log
files, you can turn this off but I have had it running like this for about 3
years and never had any problems with my last firewall (Guardian), it seems
that a big flood of DNS lookups kills the firewall service ? because this is
all that is happening. It just seems to loose the plot :)
 
Any feedback appreciated, I would like to run my reports again.
 
Phill
 
Phill Hardstaff 
MCSA, CCNA, A+, Network+, Inet+, Server+, CIW Assoc. 
Senior Support Engineer 
Secretariat of the Pacific Community 
B.P. D5 
Noumea Cedex - 98848 
New Caledonia 
  
Phone +687-260141 
Mobile +687 838091 
Fax +687-263818 
Email  phillh@xxxxxxx <mailto:phillh@xxxxxxx>  
SPC Web Page http://www.spc.int <http://www.spc.int>  
Personal Web Page http://www.hardstaff.com <http://www.hardstaff.com>  
Personal Email Phill@xxxxxxxxxxxxx <mailto:Phill@xxxxxxxxxxxxx>  
Personal Fax +1 (603) 299-5640 

Attachment: profile_ubb6.gif
Description: GIF image

Attachment: edit_ubb6.gif
Description: GIF image

Attachment: quote_ubb6.gif
Description: GIF image

Attachment: icon1.gif
Description: GIF image

Other related posts: