[isapros] Re: [ISAServer] Firewall database corruption due to power outtage
- From: "Jim Harrison" <Jim@xxxxxxxxxxxx>
- To: <isapros@xxxxxxxxxxxxx>
- Date: Mon, 10 Jul 2006 13:56:10 -0700
I think someone needs a hug...
-------------------------------------------------------
Jim Harrison
MCP(NT4, W2K), A+, Network+, PCG
http://isaserver.org/Jim_Harrison/
http://isatools.org
Read the help / books / articles!
-------------------------------------------------------
-----Original Message-----
From: isapros-bounce@xxxxxxxxxxxxx [mailto:isapros-bounce@xxxxxxxxxxxxx] On
Behalf Of Thor (Hammer of God)
Sent: Monday, July 10, 2006 11:50
To: isapros@xxxxxxxxxxxxx
Subject: [isapros] Re: [ISAServer] Firewall database corruption due to power
outtage
Actually, it doesn't steam me in the least. While I can perceive the value
of the lockdown feature, it adds no value to my environment-- I have no
problem at all disabling it. In fact, it's been disabled since I began
logging to SQL on my servers (long time now). No worries here.
The whole thing got started when I was simply trying to identify to the ISA
group that when logging to SQL, ISA's lockdown mode is "trigger happy" when
certain maintenance takes place on the server- that "maintenance" can be the
purging of "fluff" in the WebProxyLog, performing a mass update of data in
the logs, or even regular DB Maint jobs that re-index, optimize, shrink
db's, etc. It's not about trimming logs, purging logs, or anything to
"solve" ISA logging issues- it can be something that doesn't even involve
ISA logs per se.
It was the identification that, in the real-world, "ISA Logging to SQL ==
Disabled Lockdown Mode" is the inevitable result in all the cases I
examined. I didn't even say that was a bad thing- I'm quantifying it, not
qualifying it. Like I said, I don't really care about lockdown mode. I was
just pointing out that if lockdown mode was going to be considered an
important feature to enterprise customers, (which I have seen it be) then
the end user is going to require better, customizable options regarding the
parameters used when ISA logs to SQL, or else they will be forced to disable
it. You can't tout a feature as important if it will ultimately require
disabling when deployed. That's all.
The only time I got close to being "steamed" was when recommendations kept
coming back regarding how to fix the "fluff" issues. I've already fixed the
fluff issues. I said that several times. I was asking for detailed
information on the criteria ISA uses to initiate lockdown when logging to
SQL, not how to address log cruft, structure tables, or when to run
maintenance jobs. My goal was to gain information on how to build a robust,
dependable connection that would endure "standard," every-day loads, not on
how to avoid using the connection in those environments.
That's it. Done over here ;)
t
On 7/10/06 10:32 AM, "Thomas W Shinder" <tshinder@xxxxxxxxxxx> spoketh to
all:
> Why get steamed? Seems purdy easy to me.
>
> Thomas W Shinder, M.D.
> Site: www.isaserver.org
> Blog: http://blogs.isaserver.org/shinder/
> Book: http://tinyurl.com/3xqb7
> MVP -- ISA Firewalls
>
>
>
>> -----Original Message-----
>> From: isapros-bounce@xxxxxxxxxxxxx
>> [mailto:isapros-bounce@xxxxxxxxxxxxx] On Behalf Of Jim Harrison
>> Sent: Monday, July 10, 2006 12:28 PM
>> To: isapros@xxxxxxxxxxxxx
>> Subject: [isapros] Re: [ISAServer] Firewall database
>> corruption due to power outtage
>>
>> Nope - just a workaround that makes Tim all steamy.
>>
>> -------------------------------------------------------
>> Jim Harrison
>> MCP(NT4, W2K), A+, Network+, PCG
>> http://isaserver.org/Jim_Harrison/
>> http://isatools.org
>> Read the help / books / articles!
>> -------------------------------------------------------
>>
>>
>> -----Original Message-----
>> From: isapros-bounce@xxxxxxxxxxxxx
>> [mailto:isapros-bounce@xxxxxxxxxxxxx] On Behalf Of Thomas W Shinder
>> Sent: Monday, July 10, 2006 09:20
>> To: isapros@xxxxxxxxxxxxx
>> Subject: [isapros] Re: [ISAServer] Firewall database
>> corruption due to power outtage
>>
>> Then that would fix it? Right?
>>
>> Thomas W Shinder, M.D.
>> Site: www.isaserver.org
>> Blog: http://blogs.isaserver.org/shinder/
>> Book: http://tinyurl.com/3xqb7
>> MVP -- ISA Firewalls
>>
>>
>>
>>> -----Original Message-----
>>> From: isapros-bounce@xxxxxxxxxxxxx
>>> [mailto:isapros-bounce@xxxxxxxxxxxxx] On Behalf Of Thor
>>> (Hammer of God)
>>> Sent: Monday, July 10, 2006 11:15 AM
>>> To: isapros@xxxxxxxxxxxxx
>>> Subject: [isapros] Re: [ISAServer] Firewall database
>>> corruption due to power outtage
>>>
>>> Right... That's what I said... Disable lockdown.
>>>
>>> t
>>>
>>>
>>> On 7/10/06 8:29 AM, "Jim Harrison" <Jim@xxxxxxxxxxxx>
>> spoketh to all:
>>>
>>>> Like unto thusly:
>>>>
>>> http://www.microsoft.com/technet/prodtechnol/isa/2004/plan/dis
>>> ablelockdownonlo
>>>> gfailure.mspx
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> Jim Harrison
>>>> MCP(NT4, W2K), A+, Network+, PCG
>>>> http://isaserver.org/Jim_Harrison/
>>>> http://isatools.org
>>>> Read the help / books / articles!
>>>> -------------------------------------------------------
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: isapros-bounce@xxxxxxxxxxxxx
>>> [mailto:isapros-bounce@xxxxxxxxxxxxx] On
>>>> Behalf Of Thomas W Shinder
>>>> Sent: Monday, July 10, 2006 08:10
>>>> To: isapros@xxxxxxxxxxxxx
>>>> Subject: [isapros] Re: [ISAServer] Firewall database
>>> corruption due to power
>>>> outtage
>>>>
>>>> Log Failure Alert.
>>>>
>>>> Thomas W Shinder, M.D.
>>>> Site: www.isaserver.org
>>>> Blog: http://blogs.isaserver.org/shinder/
>>>> Book: http://tinyurl.com/3xqb7
>>>> MVP -- ISA Firewalls
>>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: isapros-bounce@xxxxxxxxxxxxx
>>>>> [mailto:isapros-bounce@xxxxxxxxxxxxx] On Behalf Of Thor
>> (Hammer of
>>>>> God)
>>>>> Sent: Monday, July 10, 2006 9:24 AM
>>>>> To: isapros@xxxxxxxxxxxxx
>>>>> Subject: [isapros] Re: [ISAServer] Firewall database
>>> corruption due to
>>>>> power outtage
>>>>>
>>>>> What do you mean "the alert?" What alert?
>>>>>
>>>>> t
>>>>>
>>>>>
>>>>> On 7/10/06 7:21 AM, "Thomas W Shinder"
>>> <tshinder@xxxxxxxxxxx> spoketh
>>>>> to
>>>>> all:
>>>>>
>>>>>> Hi Tim,
>>>>>>
>>>>>> But you can do that by configuring the Alert to not send
>>>>> the system into
>>>>>> lockdown.
>>>>>>
>>>>>> Thomas W Shinder, M.D.
>>>>>> Site: www.isaserver.org
>>>>>> Blog: http://blogs.isaserver.org/shinder/
>>>>>> Book: http://tinyurl.com/3xqb7
>>>>>> MVP -- ISA Firewalls
>>>>>>
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>> Sent: Monday, July 10, 2006 9:16 AM
>>>>>>> To: Adar Greenshpon; ISA-MVP; Avi Sander; Nathan Bigman
>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>> due to power
>>>>>>> outtage
>>>>>>>
>>>>>>> I guess I misunderstood. The "mentioned below" items
>> kept going
>>>>>>> back to how to remove the fluff, rather than dealing
>> with why ISA
>>>>>>> goes into lockdown so readily. I've still not seen any
>>> information
>>>>>>> on how/when/why ISA goes into lockdown when logging to
>> ISA. You
>>>>>>> had indicated that the other information was
>> incorrect, so I was
>>>>>>> standing by for that info.
>>>>>>>
>>>>>>> It doesn't matter if you put it in a different database.
>>> When the
>>>>>>> optimization plan (you know, the "normal" plan that
>>>>> rebuilds indexes,
>>>>>>> insures integrity, etc) runs, ISA goes into lockdown. Even if I
>>>>>>> waited a week in between times that I purged the data,
>> I'll still
>>>>>>> have to perform some maintenance, and the system would still go
>>>>>>> into lockdown.
>>>>>>>
>>>>>>> The solution is to disable lockdown.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> t
>>>>>>>
>>>>>>>
>>>>>>> On 7/10/06 12:13 AM, "Adar Greenshpon"
>>>>> <Adar.Greenshpon@xxxxxxxxxxxxx>
>>>>>>> spoketh to all:
>>>>>>>
>>>>>>>> Thor: as I've indicated below, we plan on improving the
>>>>> connectivity
>>>>>>>> issue you raised. Given the current SQL logging
>>>>> limitation and your
>>>>>>>> deployment, might it be possible to split the webproxy
>>>>> log into two
>>>>>>>> databases: one for historical purposes and one for current
>>>>>>> events. The
>>>>>>>> heavy-weight maintenance could be done on the former one
>>>>>>> while the later
>>>>>>>> one would be available for ISA to dump logs into.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>> Sent: Monday, July 10, 2006 1:50 AM
>>>>>>>> To: ISA-MVP; Avi Sander; Adar Greenshpon; Nathan Bigman
>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>> due to power
>>>>>>>> outtage
>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, Jim... Thank you. Sorry that I got so short, but I
>>>>> was getting
>>>>>>>> quite
>>>>>>>> frustrated trying to make the same point over and over again.
>>>>>>>>
>>>>>>>> The focus need not be on "how to trim the logs." That's
>>>>>>> not the *real*
>>>>>>>> issue. The focus needs to be: "Regarding lockdown, what
>>>>>>> factors are in
>>>>>>>> play
>>>>>>>> when ISA logs to SQL, and under what circumstances does
>>> ISA invoke
>>>>>>>> lockdown when posting to a SQL DB?"
>>>>>>>>
>>>>>>>> Let's put the padding issues *totally* out of the picture.
>>>>>>> Let's say
>>>>>>>> that
>>>>>>>> I've got TB's of storage and that I could not care less
>>>>>>> about how much
>>>>>>>> fluff
>>>>>>>> is in my WebProxyLog-- let it get as big as it wants.
>>>>>>> However, I must
>>>>>>>> back
>>>>>>>> it up. I've also experienced ISA going into lockdown
>>>>>>> during classic db
>>>>>>>> maintenance like backing up, re-indexing, optimizing size,
>>>>>>> etc. These
>>>>>>>> processes must be run regardless of the presence of
>>>>>>> "padding issues" or
>>>>>>>> not.
>>>>>>>> In fact, even if the padding problem is solved in ISA2006, in
>>>>>>>> combination with or exclusive of contributing options
>>> in SQL 2005
>>>>>>>> (which I'm migrating to as I type), ISA will still go into
>>>>>>>> lockdown when the SQL server engages in processes
>> which may lock
>>>>>>>> the db, or delay response in some way.
>>>>>>>>
>>>>>>>> Again- (and hopefully someone will listen this time) the
>>>>>>> concern speaks
>>>>>>>> to
>>>>>>>> the "real world" usability of "lockdown mode" within the
>>>>>>> enterprise. If
>>>>>>>> Microsoft intends to tout the lockdown feature of ISA
>> as a tool
>>>>>>>> enterprise clients can use to ensure evidentiary
>> integrity, then
>>>>>>>> you
>>>>>>> must address
>>>>>>>> the
>>>>>>>> mode's "survivability" during standard, everyday,
>>>>>>> real-world processes
>>>>>>>> that
>>>>>>>> must be run on the SQL servers the ISA box is logging to,
>>>>> or it will
>>>>>>>> simply
>>>>>>>> not be used. This relates to processes that may or
>> may not have
>>>>>>>> anything to do with the log files themselves.
>>>>>>>>
>>>>>>>> If nobody cares if lockdown is utilized in the
>>>>> enterprise, then this
>>>>>>>> thread
>>>>>>>> is done. If, however, the goal is supply enterprise
>>>>> customers with a
>>>>>>>> lockdown mode they can use, then user-defined connection
>>>>>>> parameters must
>>>>>>>> be
>>>>>>>> provided in order to customize the lockdown threshold to our
>>>>>>>> environments.
>>>>>>>>
>>>>>>>> T
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 7/9/06 9:28 AM, "Jim Harrison (ISA)"
>>>>> <Jim.Harrison@xxxxxxxxxxxxx>
>>>>>>>> spoketh
>>>>>>>> to all:
>>>>>>>>
>>>>>>>>> It may help in the short run, but the issue Tim raises
>>>>>>> isn't so much
>>>>>>>>> "how much data is being logged", but "how long ISA will
>>>>>>> wait while the
>>>>>>>>> logging process fails to respond before it raises an
>>>>> alert" and "I
>>>>>>>> wanna
>>>>>>>>> set this trigger point".
>>>>>>>>>
>>>>>>>>> What he's asking for (as have more than a few customers)
>>>>>>> is some way
>>>>>>>> for
>>>>>>>>> the user to specify a "almost choking on my own data"
>>>>>>> alert limit that
>>>>>>>>> could be used to move ISA logging to an alternate
>>>>> destination (SQL,
>>>>>>>>> file, bitbucket) until the primary logger responds once
>>>>>>> more. What we
>>>>>>>>> have now is a "choked on my own data" alert, which is a
>>>>>>> bit too late.
>>>>>>>>>
>>>>>>>>> In fact, I've seen one unfavorable comparison to
>>>>> <product deleted>
>>>>>>>> where
>>>>>>>>> they actually:
>>>>>>>>> 1. fail over to a backup logging process 2. monitor
>> the primary
>>>>>>>>> logger 3. fail back when the primary logger
>> reappears 4. import
>>>>>>>>> the alternate-store log data into the primary
>>>>> store as a
>>>>>>>>> low-pri part of the fail-back.
>>>>>>>>>
>>>>>>>>> Techincally, this is scriptable, except for the fact
>>>>> that our alert
>>>>>>>>> happens too late (and isn't "tweakable"). If one were
>>> willing to
>>>>>>>>> tolerate the momentary traffic block that's created today,
>>>>>>> a script is
>>>>>>>> a
>>>>>>>>> workable solution (except for the "pre-panic" factor).
>>>>>>>>>
>>>>>>>>> Jim Harrison
>>>>>>>>> SASD (ISA SE)
>>>>>>>>> If We Can't Fix It - It Ain't Broke!
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Susan Bradley, CPA aka Ebitz - SBS Rocks [MVP]
>>>>>>>>> [mailto:sbradcpa@xxxxxxxxxxx]
>>>>>>>>> Sent: Sunday, July 09, 2006 2:22 AM
>>>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
>>>>>>>>> Cc: Avi Sander; Adar Greenshpon; Nathan Bigman
>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>>>> due to power
>>>>>>>>> outtage
>>>>>>>>>
>>>>>>>>> Shouldn't depadding not cause the system to freak out
>>> and go into
>>>>>>>>> lockdown? That's his real point.. not necessarily the
>>>>>>> fact he has to
>>>>>>>> do
>>>>>>>>>
>>>>>>>>> it..but the fact that ISA is there locking itself down in
>>>>>>> the process
>>>>>>>>> ....
>>>>>>>>>
>>>>>>>>> Thor (Hammer of God) wrote:
>>>>>>>>>
>>>>>>>>>> Never mind. Not sure why I bother...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> t
>>>>>>>>>>
>>>>>>>>>> On 7/9/06 12:01 AM, "Avi Sander" <asander@xxxxxxxxxxxxx>
>>>>>>> spoketh to
>>>>>>>>> all:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi Thor -
>>>>>>>>>>>
>>>>>>>>>>> Please see
>>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/tsqlr
>>>>>>>>> ef
>>>>>>>>>>> /ts_set-set_2uw7.asp .
>>>>>>>>>>>
>>>>>>>>>>> It describes how one can toggle the ANSI_PADDING
>>>>> option. This can
>>>>>>>>>>> control how fields are padded\trimmed in SQL, but
>>> should be set
>>>>>>>> prior
>>>>>>>>> to
>>>>>>>>>>> tbl creation.
>>>>>>>>>>>
>>>>>>>>>>> - avi
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>>>>> Sent: Friday, July 07, 2006 6:23 PM
>>>>>>>>>>> To: Avi Sander; Adar Greenshpon; ISA-MVP
>>>>>>>>>>> Cc: Nathan Bigman
>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>> I think we're getting off track a little here...
>>>>>>>>>>>
>>>>>>>>>>> I appreciate the link to that "fix," but I've *already
>>>>>>> solved* the
>>>>>>>>>>> padding
>>>>>>>>>>> issue. I've been including a script for that in
>> my Blackhat
>>>>>>>> trainings
>>>>>>>>>>> for
>>>>>>>>>>> years. To that effect, simply trimming the data in
>>> the default
>>>>>>>> table
>>>>>>>>>>> isn't
>>>>>>>>>>> the best way to go, IMO. The data structure is not very
>>>>>>> efficient
>>>>>>>> to
>>>>>>>>>>> begin
>>>>>>>>>>> with (for my needs). I've created my own (better designed)
>>>>>>>> structure,
>>>>>>>>>>> and
>>>>>>>>>>> trim the data as I post into that table (and there is no
>>>>>>> "trim data"
>>>>>>>>> DSN
>>>>>>>>>>> option that I know of). But that's OK-- as an
>>> administrator, I
>>>>>>>>> *expect*
>>>>>>>>>>> that I will have to adjust logging processes to better
>>>>>>> fit my needs
>>>>>>>>>>> (just
>>>>>>>>>>> like I do with IIS logs that I post to SQL). You guys
>>>>> provide the
>>>>>>>>>>> mechanism,
>>>>>>>>>>> and I customize it to suit me. Like I said, my yearly
>>>>>>> retained data
>>>>>>>> to
>>>>>>>>>>> date
>>>>>>>>>>> is only 10 gig. No problems.
>>>>>>>>>>>
>>>>>>>>>>> What I'm saying is that *when I run the process to
>>>>> trim the data*
>>>>>>>> ISA
>>>>>>>>>>> goes
>>>>>>>>>>> into "lockdown." This whole thing got started when we
>>>>>>> were talking
>>>>>>>>>>> about
>>>>>>>>>>> lockdown mode, and how "trigger happy" it is when one is
>>>>>>> logging to
>>>>>>>>> SQL.
>>>>>>>>>>>
>>>>>>>>>>> I only brought it up in an effort to identify this:
>>>>>>>>>>> If a customer logs to SQL, there is a padding problem.
>>>>>>> The padding
>>>>>>>>>>> problem
>>>>>>>>>>> *requires* DB maintenance processes to control.
>> When the DB
>>>>>>>>> maintenance
>>>>>>>>>>> is
>>>>>>>>>>> executed against any DB of consequence (just
>> several thousand
>>>>>>>> records
>>>>>>>>>>> seems
>>>>>>>>>>> to be enough), ISA goes into lockdown, presumably
>>>>> because of the
>>>>>>>> "lag"
>>>>>>>>>>> created by server load when the SQL server goes about
>>>>> parsing out
>>>>>>>> gigs
>>>>>>>>>>> of
>>>>>>>>>>> fluff data. This is true even on my production SQL
>>>>>>> cluster (which
>>>>>>>> is
>>>>>>>>>>> kickass, mind you ;) so it's not a "hardware" problem.
>>>>>>> Thus, I was
>>>>>>>>>>> forced
>>>>>>>>>>> to disable lockdown.
>>>>>>>>>>>
>>>>>>>>>>> Therefore, IF a customer is logging to SQL, AND
>> they want use
>>>>>>>>> "lockdown"
>>>>>>>>>>> THEN better user-definable timeout parameters need to be
>>>>>>> available.
>>>>>>>>>>> That's
>>>>>>>>>>> all I was saying.
>>>>>>>>>>>
>>>>>>>>>>> Of course, since you have identified that ISA2004 EE,
>>>>> ISA2006 SE,
>>>>>>>> and
>>>>>>>>>>> ISA2006 EE properly trim data before posting it to SQL,
>>>>>>> then it all
>>>>>>>>>>> seems to
>>>>>>>>>>> be a moot point, right? ;)
>>>>>>>>>>>
>>>>>>>>>>> The main box posting web access is currently ISA 2004
>>>>> SE. But I
>>>>>>>> think
>>>>>>>>>>> I've
>>>>>>>>>>> got a copy of ISA 2004 EE that I picked up in Hong Kong
>>>>>>> along with
>>>>>>>> an
>>>>>>>>>>> old
>>>>>>>>>>> Rush CD that I'll throw on that guy and see what
>>>>>>> happens. (jk;) I'll
>>>>>>>>> let
>>>>>>>>>>> you
>>>>>>>>>>> guys know what happens.
>>>>>>>>>>>
>>>>>>>>>>> Thx
>>>>>>>>>>> t
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 7/7/06 12:29 AM, "Avi Sander" <asander@xxxxxxxxxxxxx>
>>>>>>> spoketh to
>>>>>>>>> all:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> there's also an option on the ODBC DSN : 'trim
>> fields' that
>>>>>>>> partially
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> helps,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> if i recall correctly.
>>>>>>>>>>>> I'll look into this further
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>
>>>>>>>>>>>> From: Adar Greenshpon
>>>>>>>>>>>> Sent: Fri 07/07/2006 09:07
>>>>>>>>>>>> To: Avi Sander; Thor (Hammer of God);
>>>>> isaserver@xxxxxxxxxxxxxxx
>>>>>>>>>>>> Cc: Nathan Bigman
>>>>>>>>>>>> Subject: RE: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thor,
>>>>>>>>>>>>
>>>>>>>>>>>> Per the excessive padding, see this thread:
>>>>>>>>>>>> http://forums.isaserver.org/m_140009200/tm.htm - I
>>>>> think that's
>>>>>>>> what
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> you're
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> looking for as we already heard this a few times (as
>>>>> Avi pointed
>>>>>>>> out,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> problem does not exist in ISA 2004EE or ISA 2006
>>>>>>> versions). Nathan
>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> will an
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> official KB help here?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Adar.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>
>>>>>>>>>>>> From: Avi Sander
>>>>>>>>>>>> Sent: Friday, July 07, 2006 12:27 AM
>>>>>>>>>>>> To: Thor (Hammer of God)
>>>>>>>>>>>> Cc: Adar Greenshpon
>>>>>>>>>>>> Subject: RE: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> OK - that explains it.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> This is a known SQL ODBC interfacing issue.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ISA 2004 EE and ISA2006 SE & EE moved away from ODBC
>>>>>>> and use oledb
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> interfaces
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> for SQL server logging instead. This issue does not
>>>>> exist there
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> anymore as it
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> is all trimmed by design (and throughput is also greatly
>>>>>>>> increased).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The 2 potential causes of errors i mentioned earlier in
>>>>>>> the thread
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> (buffers
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> full, 4 retries) are only relevant to the OLEDB
>>>>>>> logging, and not to
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> your ISA -
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> my bad.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> let me check with the ISA2004 SE code and get back to
>>>>> you with a
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> better
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> picture...
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -avi
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>
>>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>>>>>> Sent: Thu 06/07/2006 23:12
>>>>>>>>>>>> To: Avi Sander
>>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> These are ISA2004 SE boxes. I haven't tried with
>>>>>>> ISA2006 yet, but
>>>>>>>>> I'm
>>>>>>>>>>>> running it here at The Yeti and can check it out
>> (unless you
>>>>>>>> already
>>>>>>>>>>>> know...)
>>>>>>>>>>>>
>>>>>>>>>>>> t
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 7/6/06 2:08 PM, "Avi Sander" <asander@xxxxxxxxxxxxx>
>>>>>>> spoketh to
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> all:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hey Thor -
>>>>>>>>>>>>
>>>>>>>>>>>> What ISA version are you using? and whichSKU: SE or EE?
>>>>>>>>>>>>
>>>>>>>>>>>> -avi
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>
>>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>>>>>> Sent: Thu 06/07/2006 18:39
>>>>>>>>>>>> To: Adar Greenshpon; ISA-MVP; Avi Sander
>>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hi Adar-
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, logs going to an external SQL 2000 cluster via
>>>>>>> direct gigabit
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> switched
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> link. And log retention is permanent. Though I've
>>>>>>> only got 120
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> employees,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> we
>>>>>>>>>>>> have tight policies on Internet usage and abuse and I,
>>>>>>> like many,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> must keep
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> access logs to ensure these policies are being
>>>>>>> honored, as well as
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> for any
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> number of potential legal issues that might arise. I
>>>>>>> trust that
>>>>>>>> you
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> are not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> suggesting that ISA log retention via SQL should be
>>>>> measured in
>>>>>>>>> days,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> right?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> ;)
>>>>>>>>>>>>
>>>>>>>>>>>> Let me go into more detail about the "fluff" I
>>>>> mentioned- it is
>>>>>>>> not
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> so much
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> the "gigs of logs" as it would relate to real data- it
>>>>>>> is actual
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> "fluff:"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Extra, padded spacing in the fields posted to
>> SQL from ISA.
>>>>>>>> Though
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> you guys
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> have defined the fields in SQL as nvarchar data, there
>>>>>>> is some bad
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> mojo in
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>> logging process. Cursory review of the data in
>>>>>>> something like SQL
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Query
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Analyzer may not immediately reveal this, but binary
>>>>> analyze of
>>>>>>>> the
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> data
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> files
>>>>>>>>>>>> do.
>>>>>>>>>>>>
>>>>>>>>>>>> I've created some example text files to simplify what
>>>>>>> I am talking
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> about
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> (Attached as SQL1.txt and SQL2.txt).
>>>>>>>>>>>>
>>>>>>>>>>>> Consider the following simple query to retrieve a
>>>>> single row of
>>>>>>>>> basic
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> data
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> from the WebProxyLog as stored by ISA:
>>>>>>>>>>>>
>>>>>>>>>>>> "select
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>
>>> ClientUserName,ClientAgent,DestHost,DestHostIP,SrcNetwork,DstNetwork
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> from WebProxyLog where LogID=10133541"
>>>>>>>>>>>>
>>>>>>>>>>>> (SQL1.txt contains the results, but I'll paste them
>>>>>>> here as well)
>>>>>>>>>>>> <begin paste>
>>>>>>>>>>>> ANCHORSIGN\yomomma
>>>>>>>>>>>> PGP
>>>>>>>>>>>> update.pgp.com
>>>>>>>>>>>> 63.251.255.18 VPN Clients
>>>>>>>>>>>> External
>>>>>>>>>>>>
>>>>>>>>>>>> </end paste>
>>>>>>>>>>>>
>>>>>>>>>>>> Notice all the extra spacing; again, even though the
>>>>> fields are
>>>>>>>> type
>>>>>>>>>>>> nvarchar-- ISA is padding the data. Now, let's trim
>>>>>>> it up and see
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> what
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> happens in this query:
>>>>>>>>>>>>
>>>>>>>>>>>> "select
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>> rtrim(ClientUserName),rtrim(ClientAgent),rtrim(DestHost),rtrim(DestHos
>>>>>>>>> tI
>>>>>>>>>>> P),rt>>
>>>>>>>>>>> r
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> im(SrcNetwork),rtrim(DstNetwork) from WebProxyLog where
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> LogID=10133541"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> (SQL2.txt contains the results, but again I'll
>> paste them)
>>>>>>>>>>>> <begin paste>
>>>>>>>>>>>> ANCHORSIGN\yomomma PGP update.pgp.com
>>> 63.251.255.18
>>>>>>>> VPN
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Clients
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> External
>>>>>>>>>>>> </end paste>
>>>>>>>>>>>>
>>>>>>>>>>>> The original query results are 934 bytes. The
>>> trimmed query
>>>>>>>> results
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> are 74
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> bytes. Just for this simple query, with selected
>>>>>>> fields from one
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> record, the
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> storage requirements for this data is 12.62 times
>>>>>>> greater due to
>>>>>>>> the
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> "fluff."
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Extend that out to a full logged record, and then
>>>>> multiply that
>>>>>>>>> times
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> tens of
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> thousands. *THIS* is the 1-2 gigs of "fluff" I'm
>>>>>>> talking about on
>>>>>>>> a
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> *daily*
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> basis.
>>>>>>>>>>>>
>>>>>>>>>>>> The "required maintenance" I'm talking about is where
>>>>>>> I take daily
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> raw data
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> from the WebProxyLog, trim it up, and then post it
>>>>> into my own
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> tables. In my
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> case, the _yearly_ web proxy log data for my users is
>>>>>>> only 10 gig.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> _Weekly_
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> "raw" data in the original ISA log format averages 12
>>>>>>> gig. When
>>>>>>>> I
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> run this
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> process, I am able to retrieve data via SQA, Access
>>>>>>> front end, or
>>>>>>>>> via
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> ADODB
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> objects, yet ISA will go into lockdown mode while
>>>>> logging when
>>>>>>>> this
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> job runs
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> at night. That's why I have to turn it off.
>>>>>>>>>>>>
>>>>>>>>>>>> I hope that better defines the issue as I see it.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanksl
>>>>>>>>>>>> T
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> "Tom Shinder pities Mr. T"
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 7/5/06 2:01 AM, "Adar Greenshpon"
>>>>>>>> <Adar.Greenshpon@xxxxxxxxxxxxx>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> spoketh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> to all:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Thor,
>>>>>>>>>>>>
>>>>>>>>>>>> Just so we'll be on the same page: are you directing
>>>>>>> your ISA logs
>>>>>>>>> to
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> an
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> external machine with SQL Server (SQL 2000 or 2005)?.
>>>>>>> If so, how
>>>>>>>> do
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> you do
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> the maintenance? What's your desired log retention
>>>>>>> policy (e.g. 7
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> days)?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Per the gigs of logs you're doing, have you considered
>>>>>>> reducing it
>>>>>>>>> by
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> logging images http requests?
>>>>>>>>>>>> Adar.
>>>>>>>>>>>> __________________________________________
>>>>>>>>>>>> Adar Greenshpon | Program Manager | Microsoft ISA Server
>>>>>>>>>>>> adarg@xxxxxxxxxxxxx <mailto:adarg@xxxxxxxxxxxxx>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> <mailto:adarg@xxxxxxxxxxxxx>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> | Tel: +972.4.856.1077 | SMS/Cell: +972.54.666.4579
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>>>>>> Sent: Tuesday, July 04, 2006 8:55 PM
>>>>>>>>>>>> To: Avi Sander
>>>>>>>>>>>> Cc: Adar Greenshpon; ISA-MVP
>>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> OK- the either/or situation makes more sense. I'm
>>>>>>> sure that I'm
>>>>>>>>> not
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> hitting
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> the log buffer- the 9 second minimum window is what is
>>>>>>> getting me.
>>>>>>>>>>>>
>>>>>>>>>>>> How exactly do you define a "failure to commit?" Are
>>>>>>> you actively
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> looking
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> for
>>>>>>>>>>>> a "failure" code from SQL, or is it the lack of
>> a committed
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> transaction
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> within
>>>>>>>>>>>> the time span? I ask because if I am in the middle of DB
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> maintenance, I can
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> execute a transaction that might have to wait a
>>>>> while to get a
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> committal, but
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> SQL knows about the transaction pending and won't
>>> error out.
>>>>>>>>>>>>
>>>>>>>>>>>> The SQL table structure included in 2004 results in
>>>>> HUGE table
>>>>>>>> sizes
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> for web
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> proxy logging alone. I'm talking gigs worth of fluff
>>>>>>> per day, so
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> daily DB
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> maintenance is an absolute requirement. I have
>> never had a
>>>>>>>> logging
>>>>>>>>>>>> transaction "live" through a maintenance period
>>>>>>> without going into
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> lockdown,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> hence it being disabled on all my web proxy listener
>>>>> machines.
>>>>>>>>>>>>
>>>>>>>>>>>> I understand that the reasoning behind "lockdown" is for
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> "evidentiary"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> reasons, but that seems counter intuitive to me. The
>>>>>>> actual log
>>>>>>>> is
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> more
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> important to me than the absence of a logged event
>>>>>>> when the system
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> locks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> down.
>>>>>>>>>>>> So, not only are the services not available (though a
>>>>>>>> "fail-safe"),
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> but I
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> loose any log of any attempt in the meantime. Are we
>>>>>>> going to see
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> some more
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> "user definable" setting in regard to logging and
>>>>> lockdown? If
>>>>>>>> not,
>>>>>>>>> I
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> don't
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> see how even mid-size businesses will be able to
>> log to SQL
>>>>>>>> without
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> disabling
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> the feature.
>>>>>>>>>>>>
>>>>>>>>>>>> t
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 7/4/06 5:51 AM, "Avi Sander"
>>>>>>> <asander@xxxxxxxxxxxxx> spoketh to
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> all:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> ISA will lockdown if either of the following occurs
>>>>> (whichever
>>>>>>>> comes
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> 1st):
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> - Four consecutive failures to commit the
>>>>> data into the
>>>>>>>> DB.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Our
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> default COMMIT timeout is 30 secs. We'll retry every 3
>>>>>>> secs. So at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> minimum
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> you've got 9 secs here.
>>>>>>>>>>>> - LOG buffers are exhausted\out of memory. By
>>>>>>> default on a
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> 1GB RAM
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> machine we'll accumulate tens of thousands of recs. Is
>>>>>>> there a lot
>>>>>>>>> of
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> traffic
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> going through when this happens? How much RAM does
>>>>> this machine
>>>>>>>>> have?
>>>>>>>>>>>>
>>>>>>>>>>>> What is the event description you see in the EventViewer?
>>>>>>>>>>>>
>>>>>>>>>>>> The Log buffers can be extended some more, though not
>>>>>>> recommended
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> since can
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> use up a lot of memory.
>>>>>>>>>>>>
>>>>>>>>>>>> -avi
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> From: Adar Greenshpon
>>>>>>>>>>>> Sent: Tuesday, July 04, 2006 2:03 PM
>>>>>>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
>>>>>>>>>>>> Cc: Avi Sander
>>>>>>>>>>>> Subject: RE: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Avi - wanna play logging diva?
>>>>>>>>>>>> Generally speaking, there are four internal commit
>>>>>>> retries until
>>>>>>>> ISA
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> enters
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> lockdown. We have seen a few thousand log records in
>>>>>>> the buffers
>>>>>>>> in
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> these
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> temporary moments in our stress labs (nothing you
>>>>> could really
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> witness on a
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> normal SBS box though).
>>>>>>>>>>>>
>>>>>>>>>>>> Adar.
>>>>>>>>>>>> __________________________________________
>>>>>>>>>>>> Adar Greenshpon | Program Manager | Microsoft ISA Server
>>>>>>>>>>>> adarg@xxxxxxxxxxxxx <mailto:adarg@xxxxxxxxxxxxx>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> <mailto:adarg@xxxxxxxxxxxxx>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> | Tel: +972.4.856.1077 | SMS/Cell: +972.54.666.4579
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
>>>>>>>>>>>> Sent: Wednesday, June 28, 2006 8:26 PM
>>>>>>>>>>>> To: ISA-MVP
>>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> outtage
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> If that is what you guys are expecting to happen, I
>>>>>>> can tell you,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> *ain't* the case - not when logging to SQL on another
>>>>>>> box, anyway.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> There's
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> no "using as much memory as it can" going on. It
>>>>>>> seems to be more
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> like "I
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> did not get an acknowledgement that the last
>>> transaction was
>>>>>>>>> written,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> and
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> it's been 2 seconds, so I'm going into lockdown."
>>>>>>>>>>>>
>>>>>>>>>>>> I'm looking forward to what info you can get from
>>>>> your "logging
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> diva."
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> t
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 6/28/06 8:43 AM, "Jim Harrison (ISA)"
>>>>>>>>> <Jim.Harrison@xxxxxxxxxxxxx>
>>>>>>>>>>>> spoketh to all:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Actually, it's not a fixed value (or
>>>>>>> user-changeable), other than
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> being
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> self-regulating.
>>>>>>>>>>>> ISA logging will use as much memory as it can until
>>>>>>> it determines
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> growth is unchecked. Thus, how quickly ISA will cry
>>>>>>> foul depends
>>>>>>>>> in
>>>>>>>>>>>> part on how much free memory is available.
>>>>>>>>>>>>
>>>>>>>>>>>> I'll ask our logging diva for details, but this sums
>>>>>>> it up pretty
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> well.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Jim Harrison
>>>>>>>>>>>> SASD (ISA SE)
>>>>>>>>>>>> If We Can't Fix It - It Ain't Broke!
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
>>>>>>>>>>>> Sent: Wednesday, June 28, 2006 8:09 AM
>>>>>>>>>>>> To: ISA-MVP
>>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database
>>> corruption due to
>>>>>>>> power
>>>>>>>>>>>> outtage
>>>>>>>>>>>>
>>>>>>>>>>>> It must be about a 1k buffer, then. I repeatedly
>>> put ISA in
>>>>>>>>> lockdown
>>>>>>>>>>>> within
>>>>>>>>>>>> seconds of maintenance jobs being run, or other "data
>>>>>>> intensive"
>>>>>>>>>>>> operations
>>>>>>>>>>>> on the DB. I would start the job, in in a few
>>> seconds, get
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> disconnected
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> form my TS session because of it-- It wasn't a "one
>>>>>>> time" thing-
>>>>>>>> it
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> was
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> consistent and repeatable. That's the only reason I
>>>>> went ahead
>>>>>>>> and
>>>>>>>>>>>> disabled
>>>>>>>>>>>> lockdown.
>>>>>>>>>>>>
>>>>>>>>>>>> Is the buffer user definable somewhere? What exactly
>>>>>>> does your
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> script
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> do?
>>>>>>>>>>>>
>>>>>>>>>>>> t
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 6/28/06 7:54 AM, "Jim Harrison (ISA)"
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> <Jim.Harrison@xxxxxxxxxxxxx>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> spoketh to all:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Actually, that's not true; ISA doesn't go into
>>>>> lockdown until
>>>>>>>> the
>>>>>>>>>>>> logging buffer is filled.
>>>>>>>>>>>> I agree that a "backup logging" concept is useful.
>>>>>>>>>>>> I have a script that works for ISA 2000; it
>> shouldn't be
>>>>>>>> difficult
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> to
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> "agnosticize" it.
>>>>>>>>>>>>
>>>>>>>>>>>> Jim Harrison
>>>>>>>>>>>> Security Platform Group (ISA SE)
>>>>>>>>>>>>
>>>>>>>>>>>> If We Can't Fix It - It Ain't Broke!
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Thor (Hammer of God)
>> [mailto:thor@xxxxxxxxxxxxxxx]
>>>>>>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
>>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
>>>>>>>>>>>> Sent: Wednesday, June 28, 2006 7:34 AM
>>>>>>>>>>>> To: ISA-MVP
>>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database
>>>>> corruption due to
>>>>>>>> power
>>>>>>>>>>>> outtage
>>>>>>>>>>>>
>>>>>>>>>>>> I'd like to see some way to at least "buffer"
>> the logging
>>>>>>>> requests
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> when
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> logging to an off-box ODBC destination, like SQL.
>>>>>>> ISA is quite
>>>>>>>>>>>> intolerant
>>>>>>>>>>>> of even the slightest interruption in logging.
>>>>> But, I guess
>>>>>>>> that
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> makes
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> sense if the goal is to have "evidentiary credibility."
>>>>>>>>>>>>
>>>>>>>>>>>> It wouldn't have to be something l337 like a
>>>>> secondary local
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> logging
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> option
>>>>>>>>>>>> or anything like that-- I think being able to
>>> set a timeout
>>>>>>>> value
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> would
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> would. With larger db's, a maintenance job
>>> would kick the
>>>>>>>> server
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> into
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> lockdown, even though the SQL service is still
>> running...
>>>>>>>>>>>>
>>>>>>>>>>>> Jim, I know you wanted me to get my infrastructure
>>>>>>> layout to you
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> again
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> for
>>>>>>>>>>>> my solution to the SQL db size problems, but I
>>>>> haven't had a
>>>>>>>>> chance
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> to
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> find
>>>>>>>>>>>> that old email. I should just re-create it ;)
>>>>>>> Regardless, even
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> with
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> that
>>>>>>>>>>>> "solution" in place, I've had to disable
>>> lockdown mode when
>>>>>>>>> logging
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> to
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> SQL
>>>>>>>>>>>> for that very reason.
>>>>>>>>>>>>
>>>>>>>>>>>> t
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 6/28/06 7:19 AM, "Thomas W Shinder"
>>>>> <tshinder@xxxxxxxxxxx>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> spoketh
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> all:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> And it's a really good security and forensics decision.
>>>>>>>>>>>>
>>>>>>>>>>>> Thomas W Shinder, M.D.
>>>>>>>>>>>> Site: www.isaserver.org
>>>>>>>>>>>> Blog: http://blogs.isaserver.org/shinder/
>>>>>>>>>>>> Book: http://tinyurl.com/3xqb7 MVP -- ISA Firewalls
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Amy Babinchak
>>>>> [mailto:amy@xxxxxxxxxxxxxxxxxxxxxxxxxx]
>>>>>>>>>>>> <mailto:amy@xxxxxxxxxxxxxxxxxxxxxxxxxx%5d>
>>>>>>>>>>>> <mailto:amy@xxxxxxxxxxxxxxxxxxxxxxxxxx%5d>
>>>>>>>>>>>> Sent: Wednesday, June 28, 2006 6:28 AM
>>>>>>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
>>>>>>>>>>>> Subject: RE: [ISAServer] Firewall database
>>>>>>> corruption due to
>>>>>>>>>>>> power outtage
>>>>>>>>>>>>
>>>>>>>>>>>> I like that ISA shuts down if it can't log. It's
>>>>>>> an indicator
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> something has gone wrong.
>>>>>>>>>>>>
>>>>>>>>>>>> Amy
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Susan Bradley, CPA aka Ebitz - SBS Rocks [MVP]
>>>>>>>>>>>> [mailto:sbradcpa@xxxxxxxxxxx]
>>>>>>> <mailto:sbradcpa@xxxxxxxxxxx%5d>
>>>>>>>>>>>> <mailto:sbradcpa@xxxxxxxxxxx%5d>
>>>>>>>>>>>> Sent: Wednesday, June 28, 2006 12:12 AM
>>>>>>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
>>>>>>>>>>>> Subject: [ISAServer] Firewall database corruption
>>>>>>> due to power
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> outtage
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> With the caveat that we all know we need a good
>>>>>>> UPS but like
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> stuff
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> happens.. I've seen two folks recently have issues
>>>>>>> getting a
>>>>>>>>>>>> server back
>>>>>>>>>>>>
>>>>>>>>>>>> up have a 'dirty shutdown' due to power outtage
>>>>>>> that ended up
>>>>>>>>>>>> corrupting
>>>>>>>>>>>>
>>>>>>>>>>>> the ISA firewall database.
>>>>>>>>>>>>
>>>>>>>>>>>> Short of uninstalling and reinstalling... short of
>>>>>>> adjusting
>>>>>>>>>>>> ISA so it
>>>>>>>>>>>> won't shut down if it can't log... short of a
>>>>> better UPS...
>>>>>>>> any
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> other
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> recommendations on handling this better?
>>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>>>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in LEAVE
>>>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>>>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in LEAVE
>>>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in LEAVE
>>>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in LEAVE
>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in LEAVE
>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx In
>>>>>>>>>>>> the subject line put in LEAVE
>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>> youremailaddress
>>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> youremailaddress
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx In
>>>>>>>>>>>> the subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> youremailaddress
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> youremailaddress
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx In
>>>>>>>>>>>> the subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> youremailaddress
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> To subscribe to the list - send an email to
>>>>>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>>>> In the subject line put in JOIN
>> isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> youremailaddress
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx In
>>>>>>>>>>>> the subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> youremailaddress
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---
>>>>>>>>>> To subscribe to the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>>> In the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>> youremailaddress
>>>>>>>>>>
>>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx In the
>>>>>>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>> youremailaddress
>>>>>>>>>>
>>>>>>>>>> Don't forget the comma!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>> To subscribe to the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>> In the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>> youremailaddress
>>>>>>>>>
>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx In the
>>>>>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
>>>>>>>>> youremailaddress
>>>>>>>>>
>>>>>>>>> Don't forget the comma!
>>>>>>>>> ---
>>>>>>>>> To subscribe to the list - send an email to
>>> list@xxxxxxxxxxxxxxx
>>>>>>>>> In the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
>>>>>>>> youremailaddress
>>>>>>>>>
>>>>>>>>> To leave the list - send an email to
>>> list@xxxxxxxxxxxxxxx In the
>>>>>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
>>>>>>>> youremailaddress
>>>>>>>>>
>>>>>>>>> Don't forget the comma!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---
>>>>>>> To subscribe to the list - send an email to
>>> list@xxxxxxxxxxxxxxx In
>>>>>>> the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
>>>>>>> youremailaddress
>>>>>>>
>>>>>>> To leave the list - send an email to
>> list@xxxxxxxxxxxxxxx In the
>>>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
>>>>>>> youremailaddress
>>>>>>>
>>>>>>> Don't forget the comma!
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> All mail to and from this domain is GFI-scanned.
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>> All mail to and from this domain is GFI-scanned.
>>
>>
>>
>
>
>
All mail to and from this domain is GFI-scanned.
Other related posts:
