[isalist] Re: Serious Problems Now

  • From: "Ray Dzek" <Ray.Dzek@xxxxxxxxxxxxxxx>
  • To: <isalist@xxxxxxxxxxxxx>
  • Date: Sat, 8 Apr 2006 13:30:52 -0700

Sorry .. Win Proxy Filter.  Under Protocols, HTTP/HTTPS, Properties, 
Parameters, Application Filters.
 
The HTTPS problem is what kicked off the whole chain of events, which is why 
ISA was suspect.
 
You are correct though in the that I rarely start tearing into ISA just because 
somebody starts pointing fingers at the firewall for something they can't do on 
the internet.  ISA has been very stable in our environment.  Even after 
replacing/upgrading the Cisco firewalls for the e-commerce sites, I insisted on 
keeping ISA for our back office traffic and to support VPN and our OWA site.  
This was just one of those "perfect storm" type scenarios where all finger were 
pointing at ISA.

________________________________

From: isalist-bounce@xxxxxxxxxxxxx on behalf of Thomas W Shinder
Sent: Sat 4/8/2006 12:01 PM
To: isalist@xxxxxxxxxxxxx
Subject: [isalist] Re: Serious Problems Now


Hi Ray,
 
You need to remember that 99.987587% of the time, its not an ISA firewall 
problem, its something else. While its common to make the ISA firewall the 'sin 
eater' for everyone elses' problems, it just ain't the case.
 
That's way you need to do some troubleshooting to figure out what's going on. I 
have the same problems with network connectivity from time to time, but I don't 
start with the ISA firewall, I start with the problem. Then I use tools like:
 
* nslookup (in interactive mode)
* ping
* pathping
* tracert with and without -d switch
* ISA firewall logs
* Network Monitor (or any other packet tracer)
* ISA Alerts
* Event Viewer
* ISA firewall BPA
* Telnet
 
First step is to try to replicate the problem. Then check the ISA firewall logs 
to see what they report regarding the problem. Make sure you're logging for all 
fields in the Web proxy and Firewall logs
 
If the ISA firewall logs show that the ISA firewall isn't blocking the 
connection, then you need to figure out where in the request/response path the 
connections are being blocked. 
 
Is it a name resolution problem?
Is it a black hole router?
Is is a router loop somewhere in the path?
Is it a link failure at the ISP?
Is it a link failure of your CPE?
Is a bad switch or switch port?
Is it a bad internal router?
Is it a hardware problem on the ISA firewall device?
Is it a bad cable to a network switch, router or ISA firewall?
Is it network at 100% network utilization problem? (and if so, why? Worms? 
Warez?)
Is it a problem with the ISP's Web proxy (if they're using one)
Is it a link failure for a major provider (akamai, etc)?
Is it client problem?
Is it an authentication problem?
Is it a Group Policy problem?
 
That's just a short list of things to consider before considering a problem 
with the ISA firewall software/configuration. Its one thing if something never 
worked, but when things have worked fine before, and then they stop working, 
 
BTW -- what's "winproxy" is it this? http://www.winproxy.com/
 
HTH,
Tom
Thomas W Shinder, M.D.
Site: www.isaserver.org <http://www.isaserver.org/> 
Blog: http://blogs.isaserver.org/shinder/
Book: http://tinyurl.com/3xqb7 <http://tinyurl.com/3xqb7> 
MVP -- ISA Firewalls

 


________________________________

        From: isalist-bounce@xxxxxxxxxxxxx 
[mailto:isalist-bounce@xxxxxxxxxxxxx] On Behalf Of Ray Dzek
        Sent: Saturday, April 08, 2006 12:50 PM
        To: isalist@xxxxxxxxxxxxx
        Subject: RE: [isalist] Re: Serious Problems Now
        
        
        I may be a little "Andrew-ish" at this point.  Last week I had a 
perfectly functional ISA server.  This week, I don't know what I have.  First, 
we lost HTTPS connections.  We still have no idea why.  We previously did not 
have winproxy applied to https.  The fix was to apply winproxy, restart the 
services, and then un-apply it again and restart services.  The failure was 
such that secure NAT would not accept HTTPS at all.  You would simply get a 
time out.  No error was being generated in the logs, other than a "failure".  
No error codes, no nothing.  This wasted 2 days of my time.
         
        So since we were tearing into ISA anyway, it seemed like a good time to 
catch up on service packs for the OS and ISA.  I had not seen anything on the 
list to make me think this would be a bad thing.  And in most cases it makes 
little problems go away.  I was wrong.
         
        As explained earlier, Marketing was launching a new series of podcasts. 
 When we applied SP2, that broke iTunes which generated another post to the 
list.
         
        Next on the list was a series of stupid user tricks for which ISA was 
blamed because after the first and second problem nobody trusted ISA to be 
working properly any longer.  This wasted another 2 days.  
         
        So that brings me to my "Andrew" day.  I claimed 1/2 the internet was 
not working.  I was actually pretty close.  The issue was, and still is for 
some sites that You can go to site www.some.site with no problems.  But if you 
click on a link on that site, it breaks and goes nowhere.  So my.yahoo.com with 
all the news feeds, for instance.  The page would load fine, but if you clicked 
on one of the new items, it would just time out.  You can't log into Yahoo 
mail, msn mail etc.  The links from my RSS feeds would not work.  All other 
protocols seem to work fine.  Skype, IM, streaming radio, email, etc.  It 
appears to be directly related to http redirects, but I am not sure.
         
        I have a pretty good relationship with our upstream ISP's.  I use IM 
with the NOC on a regular basis with some of the techs there.  As it turns out, 
one of the ISP's had started getting customer complaints about the same issue. 
So the issue on my "Andrew" day is probably not ISA related.  But after the 
week I have already had, ISA was about to be replaced with a Linux iptables 
script.  I am working with both ISP's right now to see if we can isolate the 
trouble.  One of the ISP's is reporting errors on one of the T3's, but I can't 
see how a T3 transmission error could only block http redirects.  But who 
knows...
         
        BTW... did you hear the joke about the user that wrote to a firewall 
support list and asked how to open a port?
         
        I hope you all are enjoying your Saturday.

________________________________

        From: isalist-bounce@xxxxxxxxxxxxx on behalf of Jim Harrison
        Sent: Fri 4/7/2006 5:12 PM
        To: isalist@xxxxxxxxxxxxx
        Subject: [isalist] Re: Serious Problems Now
        
        

        http://www.ISAserver.org
        -------------------------------------------------------
         
        C'mon, Ray - you're starting to sound like Andrew now.
        "Half the Internet"?
        
        -----Original Message-----
        From: isalist-bounce@xxxxxxxxxxxxx [mailto:isalist-bounce@xxxxxxxxxxxxx]
        On Behalf Of Ray Dzek
        Sent: Friday, April 07, 2006 5:07 PM
        To: isalist@xxxxxxxxxxxxx
        Subject: [isalist] Serious Problems Now
        
        So .. SP2.. What the hell am I supposed to do now?
        
        Half the internet does not work.
        
        I can go to www.yahoo.com, but if you try to open yahoo mail from your
        my yahoo page, it just hangs.
        
        In the logs I see Initiated Connection then Closed Connection  over and
        over.  The traffic is not being denied.  Error is  0x0
        
        I can't get to slashdot rss feeds, etc
        
        I can seem to get to a standard www page, but anything with extended
        cookies or redirects or whatever seems to die. 
        
        Any suggestions besides spend the weekend rebuilding this thing?  I
        noticed that SP2 has no uninstall.  How nice.
        
        
        
        
        Ray Dzek
        Net Ops / Helpdesk Supervisor
        Specialized Bicycle Components
        
        
        
        All mail to and from this domain is GFI-scanned.
        
        ------------------------------------------------------
        List Archives: //www.freelists.org/archives/isalist/ 
        ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
        ISA Server Articles and Tutorials: 
http://www.isaserver.org/articles_tutorials/
        ISA Server Blogs: http://blogs.isaserver.org/
        ------------------------------------------------------
        Visit TechGenix.com for more information about our other sites:
        http://www.techgenix.com
        ------------------------------------------------------
        To unsubscribe visit http://www.isaserver.org/pages/isalist.asp
        Report abuse to listadmin@xxxxxxxxxxxxx
        
        

Other related posts: