[ldapdata] LDAP Administration Newsletter 12-18 (LDAP TroubleShooting)

From: Hallett German <hrgerman@xxxxxxxxxxxxx>
To: ldapdata@xxxxxxxxxxxxx
Date: Sat, 18 Dec 2004 17:17:02 -0500

Sincerely,
Hallett German
Alessea Software
URL: http://www.alessea.com
RSS: http://www.alessea.com/feed.xml
mail:hallett.german@xxxxxxxxxxx
-- Attached file included as plaintext by Ecartis --

LDAP: ACCESS & DATA ADMINISTRATION NEWSLETTER
12/18/04
Topics: A Catalog of Anxiety:
What Can Go Wrong With LDAP Administration

Issue Contents:

* A Catalog of Anxiety: What Can Go Wrong With LDAP Administration
* Next Time: An "Interview" with Hans Maeda of Coral Directory
* Articles and Comments Welcome
_______________________________________________________________
This newsletter is sponsored by Alessea Consulting.

Business/IT Services for small and medium businesses.
Specializing in network identity, project management, and
business development.

Visit us and read more about the Alessea difference.

URL: http://www.alessea.com
RSS: http://www.alessea.com/feed.xml
Blog: http://alessea.com/v-web/b2/
Phone: 860-346-9121
_______________________________________________________________
By Hallett German

Topic: A Catalog of Anxiety: What Can Go Wrong With LDAP Administration

[Note: Between publishing the "interview" articles, we will be running
a new series. It was inspired by a real LDAP outage caused by
a software bug.]

There are many things under the sun, both big and small, that can cause
a healthy LDAP server to fail and an LDAP Administrator to lose a good deal of
sleep. In our travels throughout the LDAP universe, we haven't found these
completely documented in one place. Here is our modest attempt to correct
this and provide suggested solutions as well.

THE HUMAN FACTOR

Whatever their skill level, all LDAP administrators will eventually
make one or more mistakes. Some of these are the following:

1) Configuration Setting Errors
Accidentally making the wrong configuration setting can halt or impair
the LDAP service process. It is possible that the setting may not have
any visible impact on system performance but may still cause unseen
problems. For this type of problem, it may take some time to resolve
because there is not an obvious error. Following change control,
and configuration management processes provides an audit trail and
notifies others when a change has taken place. Building utility scripts
to update the configuration can also reduce bad input as well.

2) Restarting Services
Sometimes, a newbie LDAP administrator will restart an LDAP service process in
attempting to solve one problem -- during the work day. But this "quick fix"
causes users to lose temporary access to LDAP services. An even worse outcome
would be if the restarted LDAP service fails to start or encounters problems
after restarting.

3) Deleting a key attribute, value, or entry.
It only takes a second to delete a key attribute and value such as the the last
record replicated or other operational attributes. Another serious loss is if
an application ID used by your HR or Network Identity system disappears.

4) Deleting an ACL or Index
Removing an Access Control List (or the equivalent for your your LDAP server)
or an
application id/application may cause an application to no longer be able to
read, write, and search LDAP attributes. If an index is deleted, then searches
may dramatically increase their session time.

5)Re-indexing
Re-indexing LDAP indices may place the directory in read-only mode. The result
is
the inability to update LDAP attributes (such as password) for a lengthy period
of time.

6) Deleting the LDAP replication between two servers
If the LDAP replication "agreement" is destroyed, then users/applications that
explicitly use the slave server will no longer have current information.
This may cause various problems (such as password changes may be out of sync.)
Depending how long it takes, the entire LDAP database may have to be rebuilt.

PLUMBING PROBLEMS - Network and Software Infrastructure

Although not directly related to LDAP errors, problems with any of the following
will impact LDAP directory operations:

Network Components:

A network component can fail at any time. This can create havoc for an LDAP
server such as "floods" where the server receives too much traffic and
eventually runs
out of connections. Or the LDAP server keeps trying to repeatedly connect
without success. This will also dramatically increase the size of the logs, so
always keep track of your disk space for such a rainy day.

It is good to have a continuous dialog with your network folks so they can tell
you how the LDAP servers are impacting the network and vice versa.

Infrastructure Software:

DNS, Time, E-Mail (SMTP, POP,IMAP, Proprietary) and other application software
may cause LDAP servers not to find each other or impact performance. Documenting
all of the infrastructure software dependencies that your LDAP servers may
help you in later troubleshooting issues.

HARDWARE/OPERATING SYSTEMS: CAN'T LIVE WITH IT, CAN'T LIVE WITHOUT IT

LDAP Servers, regardless of operating system, will periodically be susceptible
to various hardware/operating system problems:

Disk capacity:

Your LDAP logs will grow and use up disk space. (As said before, this is true
especially during outages.) Log monitoring warning should notify you as the
drive or partition reaches 90-95%. Once you have a full disk, you will have
no logs of any problems, and LDAP operations will shortly fail because there is
no space left to write to the database. The administrator will then a frantic
hour or so moving and compressing log files. The solution is aggressive log
rotation and archiving, disk space monitoring (if not already enabled), and
utilities to
list large files on a disk.

Hardware failure:
Memory, Disk, and Servers will fail at some time. The impact of failure is
minimized through use of hot spares and redundant servers. All servers should
be undergoing frequent hardware diagnostic checks to avoid such problems.

Servers exceeding capacity:
LDAP servers have become THE means to provide mission-critical authentication
for organizations. It has become a victim of its own success as an increasing
number of users and applications rely on the corporate LDAP servers to perform
their job.
However, usually companies are not doing the corresponding work of optimizing
their existing hardware/software and "right-sizing" servers for today's and
tommorrow's load. The result will be that one day when some LDAP query or
application brings the server's performance to the crawl. Then this starts to
happen everyday and users complain of having an unstable and inconsistent
performer. At least once a quarter, evaluate if your servers are adequate for
handling the current and future needs.

Operating Systems:
Operating systems can have bugs and problems under certain set of conditions
which
may impact LDAP Server performance. As new operating set releases and updates
appear, be on the lookout for features and fixes that will have a positive or
negative impact on your LDAP servers.

In a future newsletter, we will talk about the impact that LDAP server
software, applications, and security can have on your LDAP server performance
and stability.

References:
Here are some representative references:

Microsoft Site-Server LDAP Troubleshooting Guide
Good information that applies to most LDAP servers and situations.
http://www.microsoft.com/technet/prodtechnol/sscomm/reskit/ldaptsho.mspx

LDAP Errors and What They Mean
A listing of the standard LDAP error messages and what they mean.
http://www.bemsel.com/Technology/Troubleshooting/LDAP_Troubleshooting/body_ldap_troubleshooting.html

Sun - LDAP Troubleshooting
Some good things to check on UNIX LDAP servers
http://docs.sun.com/app/docs/doc/816-7511/6mdgu0h3s?a=view

IBM Notes/Domino LDAP Troubleshooting
This may apply to other LDAP servers as well
http://www-12.lotus.com/ldd/doc/domino_notes/Rnext/help6_admin.nsf/f4b82fbb75e942a6852566ac0037f284/35e2e32969a22e5985256c1d0039da76?OpenDocument

Troubleshooting Novell's eDirectory
Book with Sample Chapter
http://www.informit.com/title/0789731460

LDAP Connection with BEA WebLogic Server
Again, there may be ideas that apply across LDAP servers.
http://www.fawcette.com/weblogicpro/2004_09/magazine/columns/troubleshootersdiary/default_pf.aspx

Next Time: An "interview" with Hans Maeda of Coral Directory.

Topic: Articles and Comments Welcome

I welcome 100-800 word articles for inclusion in future
issues. Vendors and LDAP data administrators are
particularly welcome. Of course, you receive full credit and
ownership of your article. Thanks in advance for your help.

Please feel free to comment on how useful it was and what
you would like to see in the future.
Contact me at hallett.german@xxxxxxxxxxxx
______________________________________________________________
About Hal German

Hallett German has 20 years experience in a variety of
IT positions and in implementing stable infrastructures.
This includes directories/messaging architecture,
desktop support, and IT management. Hal is the founder
of the Northeast SAS Users Group and former President
of the REXX Language Association. He is the author of
three books on scripting languages. Periodically, he
writes articles on various business and IT topics.

______________________________________________________________
Contacting Hal German/Past Issues

Mail: hallett.german@xxxxxxxxxxx

Archive of the LDAP Administration Newsletter:
http://www.alessea.com/newsletters.htm
_______________________________________________________________

[ldapdata] LDAP Administration Newsletter 12-18 (LDAP TroubleShooting)

Other related posts: