[isapros] Re: [ISAServer] Firewall database corruption due to power outtage
- From: "Thomas W Shinder" <tshinder@xxxxxxxxxxx>
- To: <isapros@xxxxxxxxxxxxx>
- Date: Mon, 10 Jul 2006 12:35:31 -0500
I just wanted to see how far we could push you ;)
Thomas W Shinder, M.D.
Site: www.isaserver.org
Blog: http://blogs.isaserver.org/shinder/
Book: http://tinyurl.com/3xqb7
MVP -- ISA Firewalls
> -----Original Message-----
> From: isapros-bounce@xxxxxxxxxxxxx
> [mailto:isapros-bounce@xxxxxxxxxxxxx] On Behalf Of Thor
> (Hammer of God)
> Sent: Monday, July 10, 2006 12:33 PM
> To: isapros@xxxxxxxxxxxxx
> Subject: [isapros] Re: [ISAServer] Firewall database
> corruption due to power outtage
>
> You're going this on purpose, aren't you?
>
>
>
>
> On 7/10/06 9:19 AM, "Thomas W Shinder" <tshinder@xxxxxxxxxxx>
> spoketh to
> all:
>
> > Then that would fix it? Right?
> >
> > Thomas W Shinder, M.D.
> > Site: www.isaserver.org
> > Blog: http://blogs.isaserver.org/shinder/
> > Book: http://tinyurl.com/3xqb7
> > MVP -- ISA Firewalls
> >
> >
> >
> >> -----Original Message-----
> >> From: isapros-bounce@xxxxxxxxxxxxx
> >> [mailto:isapros-bounce@xxxxxxxxxxxxx] On Behalf Of Thor
> >> (Hammer of God)
> >> Sent: Monday, July 10, 2006 11:15 AM
> >> To: isapros@xxxxxxxxxxxxx
> >> Subject: [isapros] Re: [ISAServer] Firewall database
> >> corruption due to power outtage
> >>
> >> Right... That's what I said... Disable lockdown.
> >>
> >> t
> >>
> >>
> >> On 7/10/06 8:29 AM, "Jim Harrison" <Jim@xxxxxxxxxxxx>
> spoketh to all:
> >>
> >>> Like unto thusly:
> >>>
> >> http://www.microsoft.com/technet/prodtechnol/isa/2004/plan/dis
> >> ablelockdownonlo
> >>> gfailure.mspx
> >>>
> >>>
> >>>
> >>>
> >>> -------------------------------------------------------
> >>> Jim Harrison
> >>> MCP(NT4, W2K), A+, Network+, PCG
> >>> http://isaserver.org/Jim_Harrison/
> >>> http://isatools.org
> >>> Read the help / books / articles!
> >>> -------------------------------------------------------
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: isapros-bounce@xxxxxxxxxxxxx
> >> [mailto:isapros-bounce@xxxxxxxxxxxxx] On
> >>> Behalf Of Thomas W Shinder
> >>> Sent: Monday, July 10, 2006 08:10
> >>> To: isapros@xxxxxxxxxxxxx
> >>> Subject: [isapros] Re: [ISAServer] Firewall database
> >> corruption due to power
> >>> outtage
> >>>
> >>> Log Failure Alert.
> >>>
> >>> Thomas W Shinder, M.D.
> >>> Site: www.isaserver.org
> >>> Blog: http://blogs.isaserver.org/shinder/
> >>> Book: http://tinyurl.com/3xqb7
> >>> MVP -- ISA Firewalls
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: isapros-bounce@xxxxxxxxxxxxx
> >>>> [mailto:isapros-bounce@xxxxxxxxxxxxx] On Behalf Of Thor
> (Hammer of
> >>>> God)
> >>>> Sent: Monday, July 10, 2006 9:24 AM
> >>>> To: isapros@xxxxxxxxxxxxx
> >>>> Subject: [isapros] Re: [ISAServer] Firewall database
> >> corruption due to
> >>>> power outtage
> >>>>
> >>>> What do you mean "the alert?" What alert?
> >>>>
> >>>> t
> >>>>
> >>>>
> >>>> On 7/10/06 7:21 AM, "Thomas W Shinder"
> >> <tshinder@xxxxxxxxxxx> spoketh
> >>>> to
> >>>> all:
> >>>>
> >>>>> Hi Tim,
> >>>>>
> >>>>> But you can do that by configuring the Alert to not send
> >>>> the system into
> >>>>> lockdown.
> >>>>>
> >>>>> Thomas W Shinder, M.D.
> >>>>> Site: www.isaserver.org
> >>>>> Blog: http://blogs.isaserver.org/shinder/
> >>>>> Book: http://tinyurl.com/3xqb7
> >>>>> MVP -- ISA Firewalls
> >>>>>
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>> Sent: Monday, July 10, 2006 9:16 AM
> >>>>>> To: Adar Greenshpon; ISA-MVP; Avi Sander; Nathan Bigman
> >>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >> due to power
> >>>>>> outtage
> >>>>>>
> >>>>>> I guess I misunderstood. The "mentioned below" items
> kept going
> >>>>>> back to how to remove the fluff, rather than dealing
> with why ISA
> >>>>>> goes into lockdown so readily. I've still not seen any
> >> information
> >>>>>> on how/when/why ISA goes into lockdown when logging to
> ISA. You
> >>>>>> had indicated that the other information was
> incorrect, so I was
> >>>>>> standing by for that info.
> >>>>>>
> >>>>>> It doesn't matter if you put it in a different database.
> >> When the
> >>>>>> optimization plan (you know, the "normal" plan that
> >>>> rebuilds indexes,
> >>>>>> insures integrity, etc) runs, ISA goes into lockdown. Even if I
> >>>>>> waited a week in between times that I purged the data,
> I'll still
> >>>>>> have to perform some maintenance, and the system would still go
> >>>>>> into lockdown.
> >>>>>>
> >>>>>> The solution is to disable lockdown.
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>> t
> >>>>>>
> >>>>>>
> >>>>>> On 7/10/06 12:13 AM, "Adar Greenshpon"
> >>>> <Adar.Greenshpon@xxxxxxxxxxxxx>
> >>>>>> spoketh to all:
> >>>>>>
> >>>>>>> Thor: as I've indicated below, we plan on improving the
> >>>> connectivity
> >>>>>>> issue you raised. Given the current SQL logging
> >>>> limitation and your
> >>>>>>> deployment, might it be possible to split the webproxy
> >>>> log into two
> >>>>>>> databases: one for historical purposes and one for current
> >>>>>> events. The
> >>>>>>> heavy-weight maintenance could be done on the former one
> >>>>>> while the later
> >>>>>>> one would be available for ISA to dump logs into.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>> Sent: Monday, July 10, 2006 1:50 AM
> >>>>>>> To: ISA-MVP; Avi Sander; Adar Greenshpon; Nathan Bigman
> >>>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >> due to power
> >>>>>>> outtage
> >>>>>>>
> >>>>>>>
> >>>>>>> Yes, Jim... Thank you. Sorry that I got so short, but I
> >>>> was getting
> >>>>>>> quite
> >>>>>>> frustrated trying to make the same point over and over again.
> >>>>>>>
> >>>>>>> The focus need not be on "how to trim the logs." That's
> >>>>>> not the *real*
> >>>>>>> issue. The focus needs to be: "Regarding lockdown, what
> >>>>>> factors are in
> >>>>>>> play
> >>>>>>> when ISA logs to SQL, and under what circumstances does
> >> ISA invoke
> >>>>>>> lockdown when posting to a SQL DB?"
> >>>>>>>
> >>>>>>> Let's put the padding issues *totally* out of the picture.
> >>>>>> Let's say
> >>>>>>> that
> >>>>>>> I've got TB's of storage and that I could not care less
> >>>>>> about how much
> >>>>>>> fluff
> >>>>>>> is in my WebProxyLog-- let it get as big as it wants.
> >>>>>> However, I must
> >>>>>>> back
> >>>>>>> it up. I've also experienced ISA going into lockdown
> >>>>>> during classic db
> >>>>>>> maintenance like backing up, re-indexing, optimizing size,
> >>>>>> etc. These
> >>>>>>> processes must be run regardless of the presence of
> >>>>>> "padding issues" or
> >>>>>>> not.
> >>>>>>> In fact, even if the padding problem is solved in ISA2006, in
> >>>>>>> combination with or exclusive of contributing options
> >> in SQL 2005
> >>>>>>> (which I'm migrating to as I type), ISA will still go into
> >>>>>>> lockdown when the SQL server engages in processes
> which may lock
> >>>>>>> the db, or delay response in some way.
> >>>>>>>
> >>>>>>> Again- (and hopefully someone will listen this time) the
> >>>>>> concern speaks
> >>>>>>> to
> >>>>>>> the "real world" usability of "lockdown mode" within the
> >>>>>> enterprise. If
> >>>>>>> Microsoft intends to tout the lockdown feature of ISA
> as a tool
> >>>>>>> enterprise clients can use to ensure evidentiary
> integrity, then
> >>>>>>> you
> >>>>>> must address
> >>>>>>> the
> >>>>>>> mode's "survivability" during standard, everyday,
> >>>>>> real-world processes
> >>>>>>> that
> >>>>>>> must be run on the SQL servers the ISA box is logging to,
> >>>> or it will
> >>>>>>> simply
> >>>>>>> not be used. This relates to processes that may or
> may not have
> >>>>>>> anything to do with the log files themselves.
> >>>>>>>
> >>>>>>> If nobody cares if lockdown is utilized in the
> >>>> enterprise, then this
> >>>>>>> thread
> >>>>>>> is done. If, however, the goal is supply enterprise
> >>>> customers with a
> >>>>>>> lockdown mode they can use, then user-defined connection
> >>>>>> parameters must
> >>>>>>> be
> >>>>>>> provided in order to customize the lockdown threshold to our
> >>>>>>> environments.
> >>>>>>>
> >>>>>>> T
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 7/9/06 9:28 AM, "Jim Harrison (ISA)"
> >>>> <Jim.Harrison@xxxxxxxxxxxxx>
> >>>>>>> spoketh
> >>>>>>> to all:
> >>>>>>>
> >>>>>>>> It may help in the short run, but the issue Tim raises
> >>>>>> isn't so much
> >>>>>>>> "how much data is being logged", but "how long ISA will
> >>>>>> wait while the
> >>>>>>>> logging process fails to respond before it raises an
> >>>> alert" and "I
> >>>>>>> wanna
> >>>>>>>> set this trigger point".
> >>>>>>>>
> >>>>>>>> What he's asking for (as have more than a few customers)
> >>>>>> is some way
> >>>>>>> for
> >>>>>>>> the user to specify a "almost choking on my own data"
> >>>>>> alert limit that
> >>>>>>>> could be used to move ISA logging to an alternate
> >>>> destination (SQL,
> >>>>>>>> file, bitbucket) until the primary logger responds once
> >>>>>> more. What we
> >>>>>>>> have now is a "choked on my own data" alert, which is a
> >>>>>> bit too late.
> >>>>>>>>
> >>>>>>>> In fact, I've seen one unfavorable comparison to
> >>>> <product deleted>
> >>>>>>> where
> >>>>>>>> they actually:
> >>>>>>>> 1. fail over to a backup logging process 2. monitor
> the primary
> >>>>>>>> logger 3. fail back when the primary logger
> reappears 4. import
> >>>>>>>> the alternate-store log data into the primary
> >>>> store as a
> >>>>>>>> low-pri part of the fail-back.
> >>>>>>>>
> >>>>>>>> Techincally, this is scriptable, except for the fact
> >>>> that our alert
> >>>>>>>> happens too late (and isn't "tweakable"). If one were
> >> willing to
> >>>>>>>> tolerate the momentary traffic block that's created today,
> >>>>>> a script is
> >>>>>>> a
> >>>>>>>> workable solution (except for the "pre-panic" factor).
> >>>>>>>>
> >>>>>>>> Jim Harrison
> >>>>>>>> SASD (ISA SE)
> >>>>>>>> If We Can't Fix It - It Ain't Broke!
> >>>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Susan Bradley, CPA aka Ebitz - SBS Rocks [MVP]
> >>>>>>>> [mailto:sbradcpa@xxxxxxxxxxx]
> >>>>>>>> Sent: Sunday, July 09, 2006 2:22 AM
> >>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
> >>>>>>>> Cc: Avi Sander; Adar Greenshpon; Nathan Bigman
> >>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >>>> due to power
> >>>>>>>> outtage
> >>>>>>>>
> >>>>>>>> Shouldn't depadding not cause the system to freak out
> >> and go into
> >>>>>>>> lockdown? That's his real point.. not necessarily the
> >>>>>> fact he has to
> >>>>>>> do
> >>>>>>>>
> >>>>>>>> it..but the fact that ISA is there locking itself down in
> >>>>>> the process
> >>>>>>>> ....
> >>>>>>>>
> >>>>>>>> Thor (Hammer of God) wrote:
> >>>>>>>>
> >>>>>>>>> Never mind. Not sure why I bother...
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> t
> >>>>>>>>>
> >>>>>>>>> On 7/9/06 12:01 AM, "Avi Sander" <asander@xxxxxxxxxxxxx>
> >>>>>> spoketh to
> >>>>>>>> all:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Hi Thor -
> >>>>>>>>>>
> >>>>>>>>>> Please see
> >>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/tsqlr
> >>>>>>>> ef
> >>>>>>>>>> /ts_set-set_2uw7.asp .
> >>>>>>>>>>
> >>>>>>>>>> It describes how one can toggle the ANSI_PADDING
> >>>> option. This can
> >>>>>>>>>> control how fields are padded\trimmed in SQL, but
> >> should be set
> >>>>>>> prior
> >>>>>>>> to
> >>>>>>>>>> tbl creation.
> >>>>>>>>>>
> >>>>>>>>>> - avi
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>>>>> Sent: Friday, July 07, 2006 6:23 PM
> >>>>>>>>>> To: Avi Sander; Adar Greenshpon; ISA-MVP
> >>>>>>>>>> Cc: Nathan Bigman
> >>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>> I think we're getting off track a little here...
> >>>>>>>>>>
> >>>>>>>>>> I appreciate the link to that "fix," but I've *already
> >>>>>> solved* the
> >>>>>>>>>> padding
> >>>>>>>>>> issue. I've been including a script for that in
> my Blackhat
> >>>>>>> trainings
> >>>>>>>>>> for
> >>>>>>>>>> years. To that effect, simply trimming the data in
> >> the default
> >>>>>>> table
> >>>>>>>>>> isn't
> >>>>>>>>>> the best way to go, IMO. The data structure is not very
> >>>>>> efficient
> >>>>>>> to
> >>>>>>>>>> begin
> >>>>>>>>>> with (for my needs). I've created my own (better designed)
> >>>>>>> structure,
> >>>>>>>>>> and
> >>>>>>>>>> trim the data as I post into that table (and there is no
> >>>>>> "trim data"
> >>>>>>>> DSN
> >>>>>>>>>> option that I know of). But that's OK-- as an
> >> administrator, I
> >>>>>>>> *expect*
> >>>>>>>>>> that I will have to adjust logging processes to better
> >>>>>> fit my needs
> >>>>>>>>>> (just
> >>>>>>>>>> like I do with IIS logs that I post to SQL). You guys
> >>>> provide the
> >>>>>>>>>> mechanism,
> >>>>>>>>>> and I customize it to suit me. Like I said, my yearly
> >>>>>> retained data
> >>>>>>> to
> >>>>>>>>>> date
> >>>>>>>>>> is only 10 gig. No problems.
> >>>>>>>>>>
> >>>>>>>>>> What I'm saying is that *when I run the process to
> >>>> trim the data*
> >>>>>>> ISA
> >>>>>>>>>> goes
> >>>>>>>>>> into "lockdown." This whole thing got started when we
> >>>>>> were talking
> >>>>>>>>>> about
> >>>>>>>>>> lockdown mode, and how "trigger happy" it is when one is
> >>>>>> logging to
> >>>>>>>> SQL.
> >>>>>>>>>>
> >>>>>>>>>> I only brought it up in an effort to identify this:
> >>>>>>>>>> If a customer logs to SQL, there is a padding problem.
> >>>>>> The padding
> >>>>>>>>>> problem
> >>>>>>>>>> *requires* DB maintenance processes to control.
> When the DB
> >>>>>>>> maintenance
> >>>>>>>>>> is
> >>>>>>>>>> executed against any DB of consequence (just
> several thousand
> >>>>>>> records
> >>>>>>>>>> seems
> >>>>>>>>>> to be enough), ISA goes into lockdown, presumably
> >>>> because of the
> >>>>>>> "lag"
> >>>>>>>>>> created by server load when the SQL server goes about
> >>>> parsing out
> >>>>>>> gigs
> >>>>>>>>>> of
> >>>>>>>>>> fluff data. This is true even on my production SQL
> >>>>>> cluster (which
> >>>>>>> is
> >>>>>>>>>> kickass, mind you ;) so it's not a "hardware" problem.
> >>>>>> Thus, I was
> >>>>>>>>>> forced
> >>>>>>>>>> to disable lockdown.
> >>>>>>>>>>
> >>>>>>>>>> Therefore, IF a customer is logging to SQL, AND
> they want use
> >>>>>>>> "lockdown"
> >>>>>>>>>> THEN better user-definable timeout parameters need to be
> >>>>>> available.
> >>>>>>>>>> That's
> >>>>>>>>>> all I was saying.
> >>>>>>>>>>
> >>>>>>>>>> Of course, since you have identified that ISA2004 EE,
> >>>> ISA2006 SE,
> >>>>>>> and
> >>>>>>>>>> ISA2006 EE properly trim data before posting it to SQL,
> >>>>>> then it all
> >>>>>>>>>> seems to
> >>>>>>>>>> be a moot point, right? ;)
> >>>>>>>>>>
> >>>>>>>>>> The main box posting web access is currently ISA 2004
> >>>> SE. But I
> >>>>>>> think
> >>>>>>>>>> I've
> >>>>>>>>>> got a copy of ISA 2004 EE that I picked up in Hong Kong
> >>>>>> along with
> >>>>>>> an
> >>>>>>>>>> old
> >>>>>>>>>> Rush CD that I'll throw on that guy and see what
> >>>>>> happens. (jk;) I'll
> >>>>>>>> let
> >>>>>>>>>> you
> >>>>>>>>>> guys know what happens.
> >>>>>>>>>>
> >>>>>>>>>> Thx
> >>>>>>>>>> t
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 7/7/06 12:29 AM, "Avi Sander" <asander@xxxxxxxxxxxxx>
> >>>>>> spoketh to
> >>>>>>>> all:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> there's also an option on the ODBC DSN : 'trim
> fields' that
> >>>>>>> partially
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> helps,
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> if i recall correctly.
> >>>>>>>>>>> I'll look into this further
> >>>>>>>>>>>
> >>>>>>>>>>> ________________________________
> >>>>>>>>>>>
> >>>>>>>>>>> From: Adar Greenshpon
> >>>>>>>>>>> Sent: Fri 07/07/2006 09:07
> >>>>>>>>>>> To: Avi Sander; Thor (Hammer of God);
> >>>> isaserver@xxxxxxxxxxxxxxx
> >>>>>>>>>>> Cc: Nathan Bigman
> >>>>>>>>>>> Subject: RE: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Thor,
> >>>>>>>>>>>
> >>>>>>>>>>> Per the excessive padding, see this thread:
> >>>>>>>>>>> http://forums.isaserver.org/m_140009200/tm.htm - I
> >>>> think that's
> >>>>>>> what
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> you're
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> looking for as we already heard this a few times (as
> >>>> Avi pointed
> >>>>>>> out,
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> the
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> problem does not exist in ISA 2004EE or ISA 2006
> >>>>>> versions). Nathan
> >>>>>>> -
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> will an
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> official KB help here?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Adar.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> ________________________________
> >>>>>>>>>>>
> >>>>>>>>>>> From: Avi Sander
> >>>>>>>>>>> Sent: Friday, July 07, 2006 12:27 AM
> >>>>>>>>>>> To: Thor (Hammer of God)
> >>>>>>>>>>> Cc: Adar Greenshpon
> >>>>>>>>>>> Subject: RE: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> OK - that explains it.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> This is a known SQL ODBC interfacing issue.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> ISA 2004 EE and ISA2006 SE & EE moved away from ODBC
> >>>>>> and use oledb
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> interfaces
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> for SQL server logging instead. This issue does not
> >>>> exist there
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> anymore as it
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> is all trimmed by design (and throughput is also greatly
> >>>>>>> increased).
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> The 2 potential causes of errors i mentioned earlier in
> >>>>>> the thread
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> (buffers
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> full, 4 retries) are only relevant to the OLEDB
> >>>>>> logging, and not to
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> your ISA -
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> my bad.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> let me check with the ISA2004 SE code and get back to
> >>>> you with a
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> better
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> picture...
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> -avi
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> ________________________________
> >>>>>>>>>>>
> >>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>>>>>> Sent: Thu 06/07/2006 23:12
> >>>>>>>>>>> To: Avi Sander
> >>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> These are ISA2004 SE boxes. I haven't tried with
> >>>>>> ISA2006 yet, but
> >>>>>>>> I'm
> >>>>>>>>>>> running it here at The Yeti and can check it out
> (unless you
> >>>>>>> already
> >>>>>>>>>>> know...)
> >>>>>>>>>>>
> >>>>>>>>>>> t
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On 7/6/06 2:08 PM, "Avi Sander" <asander@xxxxxxxxxxxxx>
> >>>>>> spoketh to
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> all:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Hey Thor -
> >>>>>>>>>>>>
> >>>>>>>>>>>> What ISA version are you using? and whichSKU: SE or EE?
> >>>>>>>>>>>>
> >>>>>>>>>>>> -avi
> >>>>>>>>>>>>
> >>>>>>>>>>>> ________________________________
> >>>>>>>>>>>>
> >>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>>>>>>> Sent: Thu 06/07/2006 18:39
> >>>>>>>>>>>> To: Adar Greenshpon; ISA-MVP; Avi Sander
> >>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Hi Adar-
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes, logs going to an external SQL 2000 cluster via
> >>>>>> direct gigabit
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> switched
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> link. And log retention is permanent. Though I've
> >>>>>> only got 120
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> employees,
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> we
> >>>>>>>>>>>> have tight policies on Internet usage and abuse and I,
> >>>>>> like many,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> must keep
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> access logs to ensure these policies are being
> >>>>>> honored, as well as
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> for any
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> number of potential legal issues that might arise. I
> >>>>>> trust that
> >>>>>>> you
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> are not
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> suggesting that ISA log retention via SQL should be
> >>>> measured in
> >>>>>>>> days,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> right?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> ;)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Let me go into more detail about the "fluff" I
> >>>> mentioned- it is
> >>>>>>> not
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> so much
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> the "gigs of logs" as it would relate to real data- it
> >>>>>> is actual
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> "fluff:"
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Extra, padded spacing in the fields posted to
> SQL from ISA.
> >>>>>>> Though
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> you guys
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> have defined the fields in SQL as nvarchar data, there
> >>>>>> is some bad
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> mojo in
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> the
> >>>>>>>>>>>> logging process. Cursory review of the data in
> >>>>>> something like SQL
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> Query
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Analyzer may not immediately reveal this, but binary
> >>>> analyze of
> >>>>>>> the
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> data
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> files
> >>>>>>>>>>>> do.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I've created some example text files to simplify what
> >>>>>> I am talking
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> about
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> (Attached as SQL1.txt and SQL2.txt).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Consider the following simple query to retrieve a
> >>>> single row of
> >>>>>>>> basic
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> data
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> from the WebProxyLog as stored by ISA:
> >>>>>>>>>>>>
> >>>>>>>>>>>> "select
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>
> ClientUserName,ClientAgent,DestHost,DestHostIP,SrcNetwork,DstNetwork
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> from WebProxyLog where LogID=10133541"
> >>>>>>>>>>>>
> >>>>>>>>>>>> (SQL1.txt contains the results, but I'll paste them
> >>>>>> here as well)
> >>>>>>>>>>>> <begin paste>
> >>>>>>>>>>>> ANCHORSIGN\yomomma
> >>>>>>>>>>>> PGP
> >>>>>>>>>>>> update.pgp.com
> >>>>>>>>>>>> 63.251.255.18 VPN Clients
> >>>>>>>>>>>> External
> >>>>>>>>>>>>
> >>>>>>>>>>>> </end paste>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Notice all the extra spacing; again, even though the
> >>>> fields are
> >>>>>>> type
> >>>>>>>>>>>> nvarchar-- ISA is padding the data. Now, let's trim
> >>>>>> it up and see
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> what
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> happens in this query:
> >>>>>>>>>>>>
> >>>>>>>>>>>> "select
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> rtrim(ClientUserName),rtrim(ClientAgent),rtrim(DestHost),rtrim(DestHos
> >>>>>>>> tI
> >>>>>>>>>> P),rt>>
> >>>>>>>>>> r
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> im(SrcNetwork),rtrim(DstNetwork) from WebProxyLog where
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> LogID=10133541"
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> (SQL2.txt contains the results, but again I'll
> paste them)
> >>>>>>>>>>>> <begin paste>
> >>>>>>>>>>>> ANCHORSIGN\yomomma PGP update.pgp.com
> >> 63.251.255.18
> >>>>>>> VPN
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> Clients
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> External
> >>>>>>>>>>>> </end paste>
> >>>>>>>>>>>>
> >>>>>>>>>>>> The original query results are 934 bytes. The
> >> trimmed query
> >>>>>>> results
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> are 74
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> bytes. Just for this simple query, with selected
> >>>>>> fields from one
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> record, the
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> storage requirements for this data is 12.62 times
> >>>>>> greater due to
> >>>>>>> the
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> "fluff."
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Extend that out to a full logged record, and then
> >>>> multiply that
> >>>>>>>> times
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> tens of
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> thousands. *THIS* is the 1-2 gigs of "fluff" I'm
> >>>>>> talking about on
> >>>>>>> a
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> *daily*
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> basis.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The "required maintenance" I'm talking about is where
> >>>>>> I take daily
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> raw data
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> from the WebProxyLog, trim it up, and then post it
> >>>> into my own
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> tables. In my
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> case, the _yearly_ web proxy log data for my users is
> >>>>>> only 10 gig.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> _Weekly_
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> "raw" data in the original ISA log format averages 12
> >>>>>> gig. When
> >>>>>>> I
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> run this
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> process, I am able to retrieve data via SQA, Access
> >>>>>> front end, or
> >>>>>>>> via
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> ADODB
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> objects, yet ISA will go into lockdown mode while
> >>>> logging when
> >>>>>>> this
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> job runs
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> at night. That's why I have to turn it off.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I hope that better defines the issue as I see it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanksl
> >>>>>>>>>>>> T
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> "Tom Shinder pities Mr. T"
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 7/5/06 2:01 AM, "Adar Greenshpon"
> >>>>>>> <Adar.Greenshpon@xxxxxxxxxxxxx>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> spoketh
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> to all:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Thor,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Just so we'll be on the same page: are you directing
> >>>>>> your ISA logs
> >>>>>>>> to
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> an
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> external machine with SQL Server (SQL 2000 or 2005)?.
> >>>>>> If so, how
> >>>>>>> do
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> you do
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> the maintenance? What's your desired log retention
> >>>>>> policy (e.g. 7
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> days)?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Per the gigs of logs you're doing, have you considered
> >>>>>> reducing it
> >>>>>>>> by
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> not
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> logging images http requests?
> >>>>>>>>>>>> Adar.
> >>>>>>>>>>>> __________________________________________
> >>>>>>>>>>>> Adar Greenshpon | Program Manager | Microsoft ISA Server
> >>>>>>>>>>>> adarg@xxxxxxxxxxxxx <mailto:adarg@xxxxxxxxxxxxx>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> <mailto:adarg@xxxxxxxxxxxxx>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> | Tel: +972.4.856.1077 | SMS/Cell: +972.54.666.4579
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ________________________________
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>>>>>>> Sent: Tuesday, July 04, 2006 8:55 PM
> >>>>>>>>>>>> To: Avi Sander
> >>>>>>>>>>>> Cc: Adar Greenshpon; ISA-MVP
> >>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> OK- the either/or situation makes more sense. I'm
> >>>>>> sure that I'm
> >>>>>>>> not
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> hitting
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> the log buffer- the 9 second minimum window is what is
> >>>>>> getting me.
> >>>>>>>>>>>>
> >>>>>>>>>>>> How exactly do you define a "failure to commit?" Are
> >>>>>> you actively
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> looking
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> for
> >>>>>>>>>>>> a "failure" code from SQL, or is it the lack of
> a committed
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> transaction
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> within
> >>>>>>>>>>>> the time span? I ask because if I am in the middle of DB
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> maintenance, I can
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> execute a transaction that might have to wait a
> >>>> while to get a
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> committal, but
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> SQL knows about the transaction pending and won't
> >> error out.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The SQL table structure included in 2004 results in
> >>>> HUGE table
> >>>>>>> sizes
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> for web
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> proxy logging alone. I'm talking gigs worth of fluff
> >>>>>> per day, so
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> daily DB
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> maintenance is an absolute requirement. I have
> never had a
> >>>>>>> logging
> >>>>>>>>>>>> transaction "live" through a maintenance period
> >>>>>> without going into
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> lockdown,
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> hence it being disabled on all my web proxy listener
> >>>> machines.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I understand that the reasoning behind "lockdown" is for
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> "evidentiary"
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> reasons, but that seems counter intuitive to me. The
> >>>>>> actual log
> >>>>>>> is
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> more
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> important to me than the absence of a logged event
> >>>>>> when the system
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> locks
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> down.
> >>>>>>>>>>>> So, not only are the services not available (though a
> >>>>>>> "fail-safe"),
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> but I
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> loose any log of any attempt in the meantime. Are we
> >>>>>> going to see
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> some more
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> "user definable" setting in regard to logging and
> >>>> lockdown? If
> >>>>>>> not,
> >>>>>>>> I
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> don't
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> see how even mid-size businesses will be able to
> log to SQL
> >>>>>>> without
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> disabling
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> the feature.
> >>>>>>>>>>>>
> >>>>>>>>>>>> t
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 7/4/06 5:51 AM, "Avi Sander"
> >>>>>> <asander@xxxxxxxxxxxxx> spoketh to
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> all:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> ISA will lockdown if either of the following occurs
> >>>> (whichever
> >>>>>>> comes
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> 1st):
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> - Four consecutive failures to commit the
> >>>> data into the
> >>>>>>> DB.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> Our
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> default COMMIT timeout is 30 secs. We'll retry every 3
> >>>>>> secs. So at
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> minimum
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> you've got 9 secs here.
> >>>>>>>>>>>> - LOG buffers are exhausted\out of memory. By
> >>>>>> default on a
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> 1GB RAM
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> machine we'll accumulate tens of thousands of recs. Is
> >>>>>> there a lot
> >>>>>>>> of
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> traffic
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> going through when this happens? How much RAM does
> >>>> this machine
> >>>>>>>> have?
> >>>>>>>>>>>>
> >>>>>>>>>>>> What is the event description you see in the EventViewer?
> >>>>>>>>>>>>
> >>>>>>>>>>>> The Log buffers can be extended some more, though not
> >>>>>> recommended
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> since can
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> use up a lot of memory.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -avi
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ________________________________
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> From: Adar Greenshpon
> >>>>>>>>>>>> Sent: Tuesday, July 04, 2006 2:03 PM
> >>>>>>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> Cc: Avi Sander
> >>>>>>>>>>>> Subject: RE: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Avi - wanna play logging diva?
> >>>>>>>>>>>> Generally speaking, there are four internal commit
> >>>>>> retries until
> >>>>>>> ISA
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> enters
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> lockdown. We have seen a few thousand log records in
> >>>>>> the buffers
> >>>>>>> in
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> these
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> temporary moments in our stress labs (nothing you
> >>>> could really
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> witness on a
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> normal SBS box though).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Adar.
> >>>>>>>>>>>> __________________________________________
> >>>>>>>>>>>> Adar Greenshpon | Program Manager | Microsoft ISA Server
> >>>>>>>>>>>> adarg@xxxxxxxxxxxxx <mailto:adarg@xxxxxxxxxxxxx>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> <mailto:adarg@xxxxxxxxxxxxx>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> | Tel: +972.4.856.1077 | SMS/Cell: +972.54.666.4579
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
> >>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
> >>>>>>>>>>>> Sent: Wednesday, June 28, 2006 8:26 PM
> >>>>>>>>>>>> To: ISA-MVP
> >>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> outtage
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> If that is what you guys are expecting to happen, I
> >>>>>> can tell you,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> that
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> *ain't* the case - not when logging to SQL on another
> >>>>>> box, anyway.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> There's
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> no "using as much memory as it can" going on. It
> >>>>>> seems to be more
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> like "I
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> did not get an acknowledgement that the last
> >> transaction was
> >>>>>>>> written,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> and
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> it's been 2 seconds, so I'm going into lockdown."
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm looking forward to what info you can get from
> >>>> your "logging
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> diva."
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> t
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 6/28/06 8:43 AM, "Jim Harrison (ISA)"
> >>>>>>>> <Jim.Harrison@xxxxxxxxxxxxx>
> >>>>>>>>>>>> spoketh to all:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Actually, it's not a fixed value (or
> >>>>>> user-changeable), other than
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> being
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> self-regulating.
> >>>>>>>>>>>> ISA logging will use as much memory as it can until
> >>>>>> it determines
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> that
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> growth is unchecked. Thus, how quickly ISA will cry
> >>>>>> foul depends
> >>>>>>>> in
> >>>>>>>>>>>> part on how much free memory is available.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'll ask our logging diva for details, but this sums
> >>>>>> it up pretty
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> well.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Jim Harrison
> >>>>>>>>>>>> SASD (ISA SE)
> >>>>>>>>>>>> If We Can't Fix It - It Ain't Broke!
> >>>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
> >>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
> >>>>>>>>>>>> Sent: Wednesday, June 28, 2006 8:09 AM
> >>>>>>>>>>>> To: ISA-MVP
> >>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database
> >> corruption due to
> >>>>>>> power
> >>>>>>>>>>>> outtage
> >>>>>>>>>>>>
> >>>>>>>>>>>> It must be about a 1k buffer, then. I repeatedly
> >> put ISA in
> >>>>>>>> lockdown
> >>>>>>>>>>>> within
> >>>>>>>>>>>> seconds of maintenance jobs being run, or other "data
> >>>>>> intensive"
> >>>>>>>>>>>> operations
> >>>>>>>>>>>> on the DB. I would start the job, in in a few
> >> seconds, get
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> disconnected
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> form my TS session because of it-- It wasn't a "one
> >>>>>> time" thing-
> >>>>>>> it
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> was
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> consistent and repeatable. That's the only reason I
> >>>> went ahead
> >>>>>>> and
> >>>>>>>>>>>> disabled
> >>>>>>>>>>>> lockdown.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Is the buffer user definable somewhere? What exactly
> >>>>>> does your
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> script
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> do?
> >>>>>>>>>>>>
> >>>>>>>>>>>> t
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 6/28/06 7:54 AM, "Jim Harrison (ISA)"
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> <Jim.Harrison@xxxxxxxxxxxxx>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> spoketh to all:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Actually, that's not true; ISA doesn't go into
> >>>> lockdown until
> >>>>>>> the
> >>>>>>>>>>>> logging buffer is filled.
> >>>>>>>>>>>> I agree that a "backup logging" concept is useful.
> >>>>>>>>>>>> I have a script that works for ISA 2000; it shouldn't be
> >>>>>>> difficult
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> to
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> "agnosticize" it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jim Harrison
> >>>>>>>>>>>> Security Platform Group (ISA SE)
> >>>>>>>>>>>>
> >>>>>>>>>>>> If We Can't Fix It - It Ain't Broke!
> >>>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Thor (Hammer of God) [mailto:thor@xxxxxxxxxxxxxxx]
> >>>>>>>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
> >>>>>>> <mailto:thor@xxxxxxxxxxxxxxx%5d>
> >>>>>>>>>>>> Sent: Wednesday, June 28, 2006 7:34 AM
> >>>>>>>>>>>> To: ISA-MVP
> >>>>>>>>>>>> Subject: Re: [ISAServer] Firewall database
> >>>> corruption due to
> >>>>>>> power
> >>>>>>>>>>>> outtage
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'd like to see some way to at least "buffer" the logging
> >>>>>>> requests
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> when
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> logging to an off-box ODBC destination, like SQL.
> >>>>>> ISA is quite
> >>>>>>>>>>>> intolerant
> >>>>>>>>>>>> of even the slightest interruption in logging.
> >>>> But, I guess
> >>>>>>> that
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> makes
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> sense if the goal is to have "evidentiary credibility."
> >>>>>>>>>>>>
> >>>>>>>>>>>> It wouldn't have to be something l337 like a
> >>>> secondary local
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> logging
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> option
> >>>>>>>>>>>> or anything like that-- I think being able to
> >> set a timeout
> >>>>>>> value
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> would
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> would. With larger db's, a maintenance job
> >> would kick the
> >>>>>>> server
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> into
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> lockdown, even though the SQL service is still running...
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jim, I know you wanted me to get my infrastructure
> >>>>>> layout to you
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> again
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> for
> >>>>>>>>>>>> my solution to the SQL db size problems, but I
> >>>> haven't had a
> >>>>>>>> chance
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> to
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> find
> >>>>>>>>>>>> that old email. I should just re-create it ;)
> >>>>>> Regardless, even
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> with
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> that
> >>>>>>>>>>>> "solution" in place, I've had to disable
> >> lockdown mode when
> >>>>>>>> logging
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> to
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> SQL
> >>>>>>>>>>>> for that very reason.
> >>>>>>>>>>>>
> >>>>>>>>>>>> t
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 6/28/06 7:19 AM, "Thomas W Shinder"
> >>>> <tshinder@xxxxxxxxxxx>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> spoketh
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> to
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> all:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> And it's a really good security and forensics decision.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thomas W Shinder, M.D.
> >>>>>>>>>>>> Site: www.isaserver.org
> >>>>>>>>>>>> Blog: http://blogs.isaserver.org/shinder/
> >>>>>>>>>>>> Book: http://tinyurl.com/3xqb7 MVP -- ISA Firewalls
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Amy Babinchak
> >>>> [mailto:amy@xxxxxxxxxxxxxxxxxxxxxxxxxx]
> >>>>>>>>>>>> <mailto:amy@xxxxxxxxxxxxxxxxxxxxxxxxxx%5d>
> >>>>>>>>>>>> <mailto:amy@xxxxxxxxxxxxxxxxxxxxxxxxxx%5d>
> >>>>>>>>>>>> Sent: Wednesday, June 28, 2006 6:28 AM
> >>>>>>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> Subject: RE: [ISAServer] Firewall database
> >>>>>> corruption due to
> >>>>>>>>>>>> power outtage
> >>>>>>>>>>>>
> >>>>>>>>>>>> I like that ISA shuts down if it can't log. It's
> >>>>>> an indicator
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> that
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> something has gone wrong.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Amy
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Susan Bradley, CPA aka Ebitz - SBS Rocks [MVP]
> >>>>>>>>>>>> [mailto:sbradcpa@xxxxxxxxxxx]
> >>>>>> <mailto:sbradcpa@xxxxxxxxxxx%5d>
> >>>>>>>>>>>> <mailto:sbradcpa@xxxxxxxxxxx%5d>
> >>>>>>>>>>>> Sent: Wednesday, June 28, 2006 12:12 AM
> >>>>>>>>>>>> To: isaserver@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> Subject: [ISAServer] Firewall database corruption
> >>>>>> due to power
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> outtage
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> With the caveat that we all know we need a good
> >>>>>> UPS but like
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> stuff
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> happens.. I've seen two folks recently have issues
> >>>>>> getting a
> >>>>>>>>>>>> server back
> >>>>>>>>>>>>
> >>>>>>>>>>>> up have a 'dirty shutdown' due to power outtage
> >>>>>> that ended up
> >>>>>>>>>>>> corrupting
> >>>>>>>>>>>>
> >>>>>>>>>>>> the ISA firewall database.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Short of uninstalling and reinstalling... short of
> >>>>>> adjusting
> >>>>>>>>>>>> ISA so it
> >>>>>>>>>>>> won't shut down if it can't log... short of a
> >>>> better UPS...
> >>>>>>> any
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> other
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> recommendations on handling this better?
> >>>>>>>>>>>>
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> >>>> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in LEAVE
> >>>> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> >>>> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in LEAVE
> >>>> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> >> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in LEAVE
> >>>> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> >> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in LEAVE
> >> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> >> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in LEAVE
> >> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> >> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx In
> >>>>>>>>>>>> the subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>> youremailaddress
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> youremailaddress
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx In
> >>>>>>>>>>>> the subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> youremailaddress
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> youremailaddress
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx In
> >>>>>>>>>>>> the subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> youremailaddress
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ---
> >>>>>>>>>>>> To subscribe to the list - send an email to
> >>>>>> list@xxxxxxxxxxxxxxx
> >>>>>>>>>>>> In the subject line put in JOIN
> isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> youremailaddress
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx In
> >>>>>>>>>>>> the subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> youremailaddress
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Don't forget the comma!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ---
> >>>>>>>>> To subscribe to the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>>> In the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>> youremailaddress
> >>>>>>>>>
> >>>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx In the
> >>>>>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>> youremailaddress
> >>>>>>>>>
> >>>>>>>>> Don't forget the comma!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> ---
> >>>>>>>> To subscribe to the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>> In the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>> youremailaddress
> >>>>>>>>
> >>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx In the
> >>>>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>>>> youremailaddress
> >>>>>>>>
> >>>>>>>> Don't forget the comma!
> >>>>>>>> ---
> >>>>>>>> To subscribe to the list - send an email to
> >> list@xxxxxxxxxxxxxxx
> >>>>>>>> In the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
> >>>>>>> youremailaddress
> >>>>>>>>
> >>>>>>>> To leave the list - send an email to
> >> list@xxxxxxxxxxxxxxx In the
> >>>>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>>> youremailaddress
> >>>>>>>>
> >>>>>>>> Don't forget the comma!
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ---
> >>>>>> To subscribe to the list - send an email to
> >> list@xxxxxxxxxxxxxxx In
> >>>>>> the subject line put in JOIN isaserver@xxxxxxxxxxxxxxx,
> >>>>>> youremailaddress
> >>>>>>
> >>>>>> To leave the list - send an email to
> list@xxxxxxxxxxxxxxx In the
> >>>>>> subject line put in LEAVE isaserver@xxxxxxxxxxxxxxx,
> >>>>>> youremailaddress
> >>>>>>
> >>>>>> Don't forget the comma!
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> All mail to and from this domain is GFI-scanned.
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >
> >
> >
>
>
>
>
Other related posts:
