RE: Filtering Logging.

  • From: "Jim Harrison" <Jim@xxxxxxxxxxxx>
  • To: "[ISAserver.org Discussion List]" <isalist@xxxxxxxxxxxxx>
  • Date: Mon, 31 Oct 2005 21:27:38 -0800

ISA *does* cache log entries when the logging service is unresponsive, but as I 
said; something has to give.  It can't "cache the log entries" forever - it has 
better thing s to do with the limited RAM you gave it.

SQL *service* is designed to operate that way *given enough resources*.
SQL can only use the CPU, RAM and disk bandwidth you give it; if you manage to 
overload it with ISA logging and single-user queries, you've clearly 
overestimated the capabilities of that machine.

If you're looking to get SQL sizing recommendations, you have to know how much 
load you expect to place on the server.  If you can't express it any better 
than "just some ISA logging", then you clearly don't understand how SQL works.

Since you've been so forthcoming with the specs for the ISA and SQL in terms of 
client load, you'll understand why you're getting such specific responses.

When you build a server with infinite RAM, CPU and disk bandwidth, lemme know - 
I'd like very much to see it.

-----Original Message-----
From: William Holmes [mailto:wtholmes@xxxxxxxxxxxxxx] 
Sent: Monday, October 31, 2005 6:43 PM
To: [ISAserver.org Discussion List]
Subject: [isalist] RE: Filtering Logging.

http://www.ISAserver.org


Actually I have read this along with several other articles on the subject
including a couple posted on isaserver.org prior to posting anything. However
these articles describe work a rounds only.

I guess if you take the position that ISA is perfect in its current form then
there's no point in discussing it. Just freeze the code now and start
treating ISA like all those black box firewalls out there that "never need
updating". 

Work a rounds that simply blow of logging or reconfigure the server on the
fly during a failure to log else where miss the point in my opinion.

ISA2004 is a great product but it would still be ISA2000 (good but certainly
not the product that ISA2004 is) if the no changes necessary attitude were
taken.

Under your rules of engagement I can log but I can't use the logs for
anything useful. No queries allowed while the ISA server is running???
What is the point of logging if this is the case?

No one's "gotta lose" or at least no one should have to. ISA could cache the
entries and post them to the SQL server when it becomes available again.

SQL servers are designed to have multiple people running queries and posting
transactions with out anything getting lost. There may be a sizing issue or a
performance problem with my SQL server which are a question I asked about
earlier which according to you have no answers.

I don't think my question about logging is out of line with the purpose of
this group. I am trying to get the most out of a great product. In my case I
have a requirement for logging. I don't think shutting down logging at the
first sign of trouble should be the answer.


  

-----Original Message-----
From: Jim Harrison [mailto:Jim@xxxxxxxxxxxx] 
Sent: Monday, October 31, 2005 4:20 PM
To: [ISAserver.org Discussion List]
Subject: [isalist] RE: Filtering Logging.

http://www.ISAserver.org

I love the liberal use of "only" in questions such as these.
The SQL server is trying to service both the ISA and the user's requests and
sooner or later, somebody's gotta lose.

<not in the least bit disguised reference to online documentation that was
clearly not researched prior to posting in this list>
http://www.microsoft.com/technet/prodtechnol/isa/2004/plan/disablelockdownonl
ogfailure.mspx 
</not in the least bit disguised reference to online documentation that was
clearly not researched prior to posting in this list>
-------------------------------------------------------
   Jim Harrison
   MCP(NT4, W2K), A+, Network+, PCG
   http://isaserver.org/Jim_Harrison/
   http://isatools.org
   Read the help / books / articles!
-------------------------------------------------------
 

-----Original Message-----
From: William Holmes [mailto:wtholmes@xxxxxxxxxxxxxx] 
Sent: Monday, October 31, 2005 13:07
To: [ISAserver.org Discussion List]
Subject: [isalist] RE: Filtering Logging.

http://www.ISAserver.org

Hello,

While not trying to be a pain in the ...

When ISA is using SQL logging it is only writing records to the SQL server
correct.

Assuming a "Perfect" SQL server why should touching the Log Tables on the SQL
server have any affect on the ISA server?

I am running reports on the SQL data and running clean-up scripts after the
reports are done. At some point you have to clean up the database or it will
simply keep growing. There is no facility (at least that I am aware of) to
have the ISA server clean up when using a SQL database.

<thinly-disguised-feature-enhancement-request>

        While it is true that I have chosen SQL to log my data I think that
it      should be a "safe" choice in the sense that if my SQL server tanks
for     whatever reason that my ISA server stays alive until if runs out of
disk space to store it logs no matter what form of logging I choose.

</thinly-disguised-feature-enhancement-request>

Thanks

Bill



-----Original Message-----
From: Jim Harrison [mailto:Jim@xxxxxxxxxxxx] 
Sent: Monday, October 31, 2005 2:18 PM
To: [ISAserver.org Discussion List]
Subject: [isalist] RE: Filtering Logging.

http://www.ISAserver.org

Basic rules of thumb:
- don't go messing about in the ISA logs while ISA is using them.
- don't change the log data; use something a bit more specific than "select
*" in your SQL queries

If you're truly uninterested in outbound logging, simply disable logging for
outbound rules.
It's impossible to provide guides for SQL sizing for ISA because (as you're
seeing) it's completely dependent on the load your ISA will handle and how
much of that traffic is logged.

ISA already employs log buffering, but as with anything else in this
universe, there's an upper limit.  If you go playing silly buggers on the SQL
server while ISA is logging, you get what you ask for.

-------------------------------------------------------
   Jim Harrison
   MCP(NT4, W2K), A+, Network+, PCG
   http://isaserver.org/Jim_Harrison/
   http://isatools.org
   Read the help / books / articles!
-------------------------------------------------------
 

-----Original Message-----
From: William Holmes [mailto:wtholmes@xxxxxxxxxxxxxx] 
Sent: Monday, October 31, 2005 11:02
To: [ISAserver.org Discussion List]
Subject: [isalist] RE: Filtering Logging.

http://www.ISAserver.org

Hello,

I can accept this. The problem is the shear volume of the logging that is
being sent to my SQL server and the nasty problem of ISA hanging when there
is a problem with the SQL server.

I see a solution to this at:
http://www.isaserver.org/pages/article_p.asp?id=346

I didn't actually have my SQL server go down, I was running a clean-up script
on the log table to clean up old and unwanted data. During this run my ISA
server stopped responding. 

Obviously my SQL server needs to be configured differently to handle the
load. I have approx 2 million entries/day.

Are there any guides on sizing a SQL server for ISA logging? I know what
doesn't work. 

I think for reliability SQL logging should have a significant local buffering
capability on the ISA server. If a background process could deal with the
transfer of data to the SQL server a leave ISA working in the event of a
communications/SQL Server problem that has the potential for a higher level
of reliability.

Bill

-----Original Message-----
From: Jim Harrison [mailto:Jim@xxxxxxxxxxxx] 
Sent: Monday, October 31, 2005 11:54 AM
To: [ISAserver.org Discussion List]
Subject: [isalist] RE: Filtering Logging.

http://www.ISAserver.org

You're wasting your and ISA's time with this.
It will actually take more time to sort out what specific requests need to be
logged than it will to filter them out in a SQL query.

-------------------------------------------------------
   Jim Harrison
   MCP(NT4, W2K), A+, Network+, PCG
   http://isaserver.org/Jim_Harrison/
   http://isatools.org
   Read the help / books / articles!
-------------------------------------------------------

________________________________________
From: William Holmes [mailto:wtholmes@xxxxxxxxxxxxxx] 
Sent: Monday, October 31, 2005 08:00
To: [ISAserver.org Discussion List]
Subject: [isalist] Filtering Logging.

http://www.ISAserver.org
Hello,
 
I have a couple of questions about logging.
 
 
Background:  The ISA server in question is used exclusively for web proxying
and web publishing.  I would like to log all inbound (Web Publishing) traffic
to SQL. I don't wish to log outbound (web proxy) traffic.  Due to the amount
of traffic being generated I would like to filter some of it out right at the
ISA server level rather than swamp an SQL table with stuff I don't really
need information on.
 
For instance I could care less about image (.jpg, .gif, .png ....) files that
are part of the site and I would prefer that they never get logged.
 
So is there a way to accomplish filtering what gets logged. I know I can shut
off logging for specific rules have how about for specific types of content
(its not clear how I could setup a web publishing rule that would server
*.gif ...  from anywhere in a published path.
 
I also can't see where to shut off outbound web proxy logging. I hope I'm
just missing that one due to fatigue.
 
Thanks
 
Bill
------------------------------------------------------
List Archives: http://www.webelists.com/cgi/lyris.pl?enter=isalist
ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
ISA Server FAQ: http://www.isaserver.org/pages/larticle.asp?type=FAQ
------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
------------------------------------------------------
You are currently subscribed to this ISAserver.org Discussion List as:
jim@xxxxxxxxxxxx
To unsubscribe visit http://www.webelists.com/cgi/lyris.pl?enter=isalist
Report abuse to listadmin@xxxxxxxxxxxxx 

All mail to and from this domain is GFI-scanned.


------------------------------------------------------
List Archives: http://www.webelists.com/cgi/lyris.pl?enter=isalist
ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
ISA Server FAQ: http://www.isaserver.org/pages/larticle.asp?type=FAQ
------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
------------------------------------------------------
You are currently subscribed to this ISAserver.org Discussion List as:
wtholmes@xxxxxxxxxxxxxx
To unsubscribe visit http://www.webelists.com/cgi/lyris.pl?enter=isalist
Report abuse to listadmin@xxxxxxxxxxxxx

------------------------------------------------------
List Archives: http://www.webelists.com/cgi/lyris.pl?enter=isalist
ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
ISA Server FAQ: http://www.isaserver.org/pages/larticle.asp?type=FAQ
------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
------------------------------------------------------
You are currently subscribed to this ISAserver.org Discussion List as:
jim@xxxxxxxxxxxx
To unsubscribe visit http://www.webelists.com/cgi/lyris.pl?enter=isalist
Report abuse to listadmin@xxxxxxxxxxxxx

All mail to and from this domain is GFI-scanned.


------------------------------------------------------
List Archives: http://www.webelists.com/cgi/lyris.pl?enter=isalist
ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
ISA Server FAQ: http://www.isaserver.org/pages/larticle.asp?type=FAQ
------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
------------------------------------------------------
You are currently subscribed to this ISAserver.org Discussion List as:
wtholmes@xxxxxxxxxxxxxx
To unsubscribe visit http://www.webelists.com/cgi/lyris.pl?enter=isalist
Report abuse to listadmin@xxxxxxxxxxxxx

------------------------------------------------------
List Archives: http://www.webelists.com/cgi/lyris.pl?enter=isalist
ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
ISA Server FAQ: http://www.isaserver.org/pages/larticle.asp?type=FAQ
------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
------------------------------------------------------
You are currently subscribed to this ISAserver.org Discussion List as:
jim@xxxxxxxxxxxx
To unsubscribe visit http://www.webelists.com/cgi/lyris.pl?enter=isalist
Report abuse to listadmin@xxxxxxxxxxxxx

All mail to and from this domain is GFI-scanned.


------------------------------------------------------
List Archives: http://www.webelists.com/cgi/lyris.pl?enter=isalist
ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
ISA Server FAQ: http://www.isaserver.org/pages/larticle.asp?type=FAQ
------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
------------------------------------------------------
You are currently subscribed to this ISAserver.org Discussion List as:
wtholmes@xxxxxxxxxxxxxx
To unsubscribe visit http://www.webelists.com/cgi/lyris.pl?enter=isalist
Report abuse to listadmin@xxxxxxxxxxxxx

------------------------------------------------------
List Archives: http://www.webelists.com/cgi/lyris.pl?enter=isalist
ISA Server Newsletter: http://www.isaserver.org/pages/newsletter.asp
ISA Server FAQ: http://www.isaserver.org/pages/larticle.asp?type=FAQ
------------------------------------------------------
Visit TechGenix.com for more information about our other sites:
http://www.techgenix.com
------------------------------------------------------
You are currently subscribed to this ISAserver.org Discussion List as: 
jim@xxxxxxxxxxxx
To unsubscribe visit http://www.webelists.com/cgi/lyris.pl?enter=isalist
Report abuse to listadmin@xxxxxxxxxxxxx

All mail to and from this domain is GFI-scanned.



Other related posts: