Web Client Requests

  • From: "Greg Mulholland" <greg@xxxxxxxxxxxxxx>
  • To: <isalist@xxxxxxxxxxxxx>
  • Date: Sat, 28 Jan 2006 07:37:34 +1100

Hey guys, im forwarding this message on behalf of Jim. He posted it to another 
list and true to form it was too good an explanation not to impart on the 
masses (or the cheesemakers).
 
This traces the path of your IE (or other) http requests and explains why you 
will always see anonymous requests in your web logs. Thanks Jim
 
Greg Mulholland

>>>>>>>>>>>>>>

Correct - all web clients do exactly that.
This is also why the logs will forever contain anonymous requests even
if all you allow are authenticated connections, because ISA will log
those denied anonymous requests.

What you can't tell from the logs is what happens after that in detail.
This requires a bit of Netmon (or Ethereal, if you swing that way)
sleuthing.

Here's the bouncing ball:

** Client, he say:
GET http://www.isaserver.org/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, application/vnd.ms-excel,
application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET
CLR 1.1.4322; InfoPath.1)
Host: www.isaserver.org
Proxy-Connection: Keep-Alive

** ISA, she say:
HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires
authorization to fulfill the request. Access to the Web Proxy service is
denied.  )
Via: 1.1 HEARTOFGOLD
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 4113 

..note - the ISA in this case (as in yours, probably) logged this
request as anonymous and responded saying that it allowed three
authentication methods: Negotiate, Kerberos and NTLM.  These are the
default auth methods for any ISA installation (including SBS).

** Client, he say:
GET http://www.isaserver.org/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, application/vnd.ms-excel,
application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET
CLR 1.1.4322; InfoPath.1)
Host: www.isaserver.org
Proxy-Connection: Keep-Alive
Proxy-Authorization: NTLM
TlRMTVNTUAABAAAAB7IIogQABAAzAAAACwALACgAAAAFASgKAAAAD0ZPUkRQUkVGRUNUSE9N
RQ==

Note that the client chose NTLM auth and passed the first part of the
handshake in Base-64 encoding.  Not to worry, this isn't like Basic,
which is base-64 encoded plain text; this is base-64 encoded encrypted
information.  ISA also logs this request as anonymous.

** ISA, she say:
HTTP/1.1 407 Proxy Authentication Required ( Access is denied.  )
Via: 1.1 HEARTOFGOLD
Proxy-Authenticate: NTLM
TlRMTVNTUAACAAAACAAIADgAAAAFgomiWWcfZe6QNCsAAAAAAAAAALQAtABAAAAABQLODgAA
AA9IAE8ATQBFAAIACABIAE8ATQBFAAEAFgBIAEUAQQBSAFQATwBGAEcATwBMAEQABAAiAGgA
bwBtAGUALgBqAGEAbABvAGoAYQBzAGgALgBvAHIAZwADADoAaABlAGEAcgB0AG8AZgBnAG8A
bABkAC4AaABvAG0AZQAuAGoAYQBsAG8AagBhAHMAaAAuAG8AcgBnAAUAIgBoAG8AbQBlAC4A
agBhAGwAbwBqAGEAcwBoAC4AbwByAGcAAAAAAA==
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 0  

Note that ISA also passed some NTLM data back to the client - this is
part and parcel to NTLM authentication even outside of HTTP

** Client, he say:
GET http://www.isaserver.org/ HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, application/vnd.ms-excel,
application/vnd.ms-powerpoint, application/msword, */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;.NET
CLR 1.1.4322; InfoPath.1)
Host: www.isaserver.org
Proxy-Connection: Keep-Alive
Proxy-Authorization: NTLM
TlRMTVNTUAADAAAAGAAYAG4AAAAYABgAhgAAAAgACABIAAAACAAIAFAAAAAWABYAWAAAAAAA
AACeAAAABYKIogUBKAoAAAAPSABPAE0ARQBKAGkAbQBIAEYATwBSAEQAUABSAEUARgBFAEMA
VABunrbKxTfLxwAAAAAAAAAAAAAAAAAAAABNhP8BkKK3ZR1MXfC2h14+Q4IQaVlWRH8=


Note that the client passes the remaining part of the NTLM handshake -
if ISA can resolve the credentials passed by the client during this
process, all will be FD&H.

** ISA, she say:
HTTP/1.1 200 OK
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Content-Length: 40936
Via: 1.1 HEARTOFGOLD
Date: Fri, 27 Jan 2006 05:49:15 GMT
Content-Type: text/html
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Set-Cookie: ASPSESSIONIDCCRRSRBC=EIBLFICAIMCPFBFCEKFFKBEA; path=/
Cache-control: private

This is where access is allowed (200 response).

You should note that I haven't included anything that may have been
passed in the HTTP body - it's not important to this discussion and only
makes for an unweildy thread.

--------------------------------------------
Jim Harrison
MCP(NT4, W2K), A+, Network+, PCG
http://isaserver.org/Jim_Harrison/
http://isatools.org
Read the help / books / articles!


Other related posts: