At 1:02 we believe all users were kicked out of slx web (Lan users not affected).  Below is our fact gathering.  Comments welcome - What are these two very short entries in the event viewer?

results from the event viewer:

1:02:52 - Not authenticated.
1:03:00 - DB_SEC_E_AUTH_FAILED
Two very strange messages, never seen this, gives no details. Out of 14K logs, these only occur once. These are the complete logs, not a snippet of each.

1:03 - 1:11 Users trying to log in, I see password failed messages. I and other users were presented with the log in screen. Some users reported system hang or timeout after, Some would get invalid userid/pw.  No one could log in.

1:11 to 1:21 No slx logs.
1:21 - IIS restarted
1:22 System back up.

You need to pay attention to how much Memory your Instance is consuming.

Also, you may want to consider running multiple Instances per Server if you aren't (better utilize resources on the servers most of the times).

Back to topic, in my experience, once the Instance reaches a certain amount of Memory usage (over one Gig, it could become unstable). It may be a good idea to restart the Instance every day (use the App Pool recycling settings).

Raul,

Thanks very much.

We've learned that lesson.  We watch memory very closely and monitor (alarms) via nagios.  We're quite sure we did not hit a memory limit.  We do recycle daily.