RE: How I spent my Christmas vacation
- From: "John T \(Lists\)" <johnlist@xxxxxxxxxxxxxxxxxxx>
- To: "'[ISAserver.org Discussion List]'" <isalist@xxxxxxxxxxxxx>
- Date: Wed, 4 Jan 2006 00:50:20 -0800
Replying to several posts and some of my replies may be mute by others that
have already responded: (NOTE: My responses are not against any one at all.
I have seen otherwise experts on most e-mail subjects take the wrong stance
on accepting all e-mail and then rejecting. If I offend any one, I have no
intention too. We all need to help one another so that we can become
stronger in our combined fight with the bad guys.)
(Dr Tom) You bet! The DNS stuff can drive you crazy. That's one nice thing
about the Exchange SMTP service, that you can disable NDR's. You can't do
that with the IIS SMTP service. I'm hoping that the Longhorn IIS SMTP
service will allow for the same functionality.
There is some confusion as to the generation of NDR's. Normally, the only
time a receiving SMTP server will generate a NDR is if AFTER it receives the
message, it is unable to deliver the message to the final destination. This
is for the most part concerning relay or gateway SMTP servers. 99% of the
time, the NDR is generated by the sending e-mail server when it is unable to
complete the passing of the message to the receiving SMTP server, whether
through permanent error such as not a valid user or account disabled to
temporary errors such as user over quote or disk error and so forth. Where
the confusion comes from is not in the generation of NDR's, but in the
receiving e-mail server being poorly configured to receive all, and then
reject. The biggest abuser of this is Yahoo.com. The second biggest abuse is
a Default Exchange server. If receiving e-mail servers were properly
configured to reject an incoming message that it would be unable to deliver
for invalid user and other such code 5 permanent failures, that would
greatly clear up the confusion.
(Joseph) That sounds like my experience when I configured GFI MailEssentials
to "generate NDR" on my Bayesian filter.
BAD BAD BAD. By generating an NDR based on anti-spam or anti-virus
scanning/filtering, you are in essence adding to the spam problem. Most of
today's spam and virus laden e-mails are from forged from addresses. As
such, you are then sending NDR's to a forged address, often times an
unsuspecting victim.
(Joseph) In my case, I do filter Recp not in AD, but I read GFI's manual
which suggested that sending the NDR to spammers would simply convince them
to take the email address off of their list. Even IF that were true (and
it's not) the cost was too high.
THIS IS ABSOLUTLE STUPIDITY! I dare some one from GFI to back this one up. I
am more than willing to discuss their suggestion about generating NDR's. I
do not know whether to cry at the lack of understanding or laugh at the
foolishness.
(Dr. Tom)No. The inbound SMTP relays are in a DMZ away from the core
critical assets, such as Exchange, SharePoint and Active Directory. The
relays accept mail only for my domains, and then scrub for viruses and spam.
While the LDAP queries would prevent this issue, I don't consider the
inbound relays as part of the same security zone as the Exchange Servers.
Why? Because the spam whackers are 1. Internet facing, and 2. must accept
anonymous inbound connections.
But that does not mean a inbound relay or gateway server can not be user
aware. There are ways of doing it, and in today's environment it MUST be
done. And yes, you do it by LDAP query but it does not have to be real time,
it could be a scheduled task updating a static list.
(Dr. Tom) I don't think it's a false sense of security. The design was not
perfect from a configuration point of view, not a security point of view.
Internet facing hosts that accept anonymous inbound connections should never
be located on the same security zone as core information assets. I pity the
fool who thinks otherwise :))
A receiving e-mail server is required by RFC to accept e-mail from <>
(anonymous) for valid users. There are many legit reasons for this. A
properly configured receiving e-mail server with appropriate anti-spam and
anti-virus scanning/filtering has not security or otherwise problem with
this.
(Dr. Tom) So, you allow LDAP queries from hosts on an anonymous access DMZ.
How do you mitigate the security issues involved with that. Yes, I know the
convention wisdom in some circles say don't accept mail to non-existing
accounts, but then you have to allow LDAP from a very low security zone. A
very poor compromise.
But an LDAP query can be restricted to what it can access.
(Dan) 1. I have to send NDRs out to people sending in mis-typed addresses,
we deal a lot with get general public, people make typos on e-mail addresses
all the time. Without the NDRs, many people would send e-mail and "assume"
it went through, and plan their activites according to those assumptions.
We don't know if the originating addresses are valid until we attempt to
send the NDR.
Incorrect thought process. If you do not accept the invalidly addressed
message in the first place, the burden is on the sending e-mail server where
it should be. That server would then generate and send an NDR back to the
sender.
(Dan) 2. Due to the wide variety of SMTP servers connecting to us, we cannot
"require" them to use a certain type of protocol just to send us e-mail.
Thus, we allow everything to come in, and then deal with the results. Too
many people in the education industry run the cheapest software they can
get, whether it is freeware or stuff that is 10-15 years old it doesn't
matter. As long as it is free.
What protocol, SMTP? Sorry, there is a standard and all must follow.
Connect, handshake, mail from, mail to, data. If mail to is not a known
valid address, reject.
(Dan) 3. Unfortunately, no-one can identify spammers by their e-mail
address or originating server, so it is impossible to tell if we're sending
e-mail to spammers or not.
Most likely, you are sending to forged and possibly unsuspecting "victims."
If, and that is a big if, you are sending back to a valid address, you just
confirmed information he/she wants. :))
(Dan) The proposed backscatter solution is just a dream. While I agree that
it IS a problem, and that there are several ways around it, there is no
"practical" solution at this time. Unless we can get EVERYONE running
completely compatible DNS servers, it will remain an illusive dream. In the
meantime, we contribute to the e-mail backscatter problem daily not by
choice but by necessity. Blocking e-mail that doesn't come from a
"compatible" server is entirely out of the question for us right now.
Not a dream at all, just do not accept for a non-valid address. And what
does this have to do with DNS servers? Nothing. Nothing except the extra
load you are placing on them trying to find records for domains that may or
may not exist, and which if exist may be constructed to cause DNS issues.
(Greg) What I did was write a script that runs on a windows box internally,
that yoinks all smtp addresses out of the AD for given domain names (like
all krystaltek.com etc) and compiles a text file which is then scp'ed to the
postfix box. A cron job on the postfix box picks this up and sticks it in
the right place (/etc/postfix/valid_recips) and postmaps it.
Yes, this is the best way and one that I employ on my gateway servers and
many others do as well. A script runs on a computer on the LAN where the
receipeint e-mail server is to gather valid addresses. That same script then
copies the resulting file to a central location whether by xcopy or ftp
depending on where it is and what is between the locations. Another script
at the central location then combines the various files into a master file
that holds all valid addresses for all clients and then does what is needed
to update the gateway SMTP servers.
(Dan) I don't have to send an e-mail to helpmeunderstandsmtp@xxxxxxxxxxxxx
to tell you that the response will come from my server. I'm running a
standards (RFC) compliant server. However, this is not the case in many of
the e-mail we receive on a daily basis. Our server gets about 500-1000
e-mails a day that require an NDR being sent. Occasionally I go through
these NDR e-mails to see what is being rejected. I keep hoping that
everything that is sent out is a result of spam, but I keep finding valid
NDRs in there. If everything was as perfect as you imply, my server would
never have to send out those messages!
You are mixing up 2 different things. If you send a message to an invalid
address, it is your server that will generate the NDR back to you if the
receiving SMTP server is correctly configured. However, on incoming e-mail
to your server, if you properly and correctly rejected during the envelope a
message to other than a valid recipient on your server, then your server
would not have a need to then create an NDR. If you are finding valid NDR's,
you need to investigate the reason. If they are for such things a user over
quote, then that is not for you to worry about. What valid NDR's are you
seeing?
(Dan) Just to appease you, I did your "Google: email backspatter" search,
but it only showed me exactly what I've been trying to describe to you. The
ability of SMTP servers to send/relay information with little or not
authentication is exactly the reason why many big companies considered
backing the e-mail postage idea. There is no good solution to the problem.
The solution proposed in many of results of that query all dealt with SMTP
servers being compliant with a set standard, rejecting all traffic that
failed to pass those tests. An NDR coming from the "sending" SMTP server
only happens when two compatible systems talk to each other.
Incorrect. An NDR coming from the sending SMTP server only goes to the
sender. The compatibility is the SMTP protocol. In fact, SMTP is probably
one of the simplest protocols there as, as you can even use telnet to create
an entire session. The SMTP server behind either the sending or receiving
could care less about what make or model they are as long as they talk SMTP
protocol.
(Dan) Sorry for the DNS confusion, I meant to say SMTP. But yes, rejecting
e-mail at the SMTP level from servers that don't or can't authenticate is
the same as blocking from non-compatible SMTP servers. That is what the
results of your Google answer-to-everything search suggest. They propose
authenticating the sender, encrypting the transaction, etc... In short,
putting more of a responsibility on the sending SMTP server. Again, this
"only" works with "current" standards (RFC) complaint SMTP servers. By not
sending out NDRs, you've cut off error messages to people that sent you a
message and think that you got it. This, in itself isn't a "huge" problem,
but it all depends on who you expect to receive e-mail from.
Oops, terminology confusion here. The only authentication ever going on is
from the sender to the sender's SMTP server. This is a must. It is sad to
note that many SMTP servers out there do not require their senders to
authenticate, but that is a whole other discussion. And once again, if the
receiving server never accepted the message to an invalid address in the
first place, your claim of must send out an NDR to inform of non-deliver
vanishes.
(Dan) I didn't misunderstand you, I just thing you're living in a fantasy
world and think everyone is running the latest and greatest software
(Freeware or not) on their SMTP servers. Yes, in a perfect world wouldn't
need to send out NDRs, but we don't live in a perfect world. Turn off your
NDR sending if you want to, since you won't see the results it will seem (to
you at least) to be the perfect solution.
Not perfect Dan, correct. Do not receive e-mail for non-valid addresses!
(Honorable Jim) When "NDR" means "respond to the sending host if possible"
instead of "send to the address in the 'from:' data field", then NDR might
once again be a useful mechanism. Until then, it's nothing more or less
than a way to invite bandwidth abuse.
Very nicely put.
(Greg) Just reject the mail at the gateway and let the sending server be
responsible for generating the NDR. not to be wasting my bandwidth please!
Exactly what should be done.
People, remember this one thing: If the war on spam and viruses were as
simple as some believe, it would have been over already.
John T
eServices For You
Other related posts:
