The initial problem was the root mount ( / ) filling up. After they
cleared up that issue, we found that we could no longer log in to the
databases. We tried bouncing the databases and that made no difference.
The sys admins couldn't find a reason on their end and we couldn't find
anything in the logs prior to the reboot. The only thing we saw in logs
after the reboot was that it wouldn't recognize the ASM disk. "Reboot"
seems to be their stock answer to solve all problems.
On Mon, Jan 10, 2022 at 12:55 PM Andy Klock <andy@xxxxxxxxxxxxx> wrote:
Hi Sandy, that sounds painful :(
On Monday, January 10th, 2022 at 2:36 PM, Sandra Becker <
sbecker6925@xxxxxxxxx> wrote:
We had some issues with a database server and the 9 databases stopped
working. The sys admins suggested a reboot, which we did. GRID/ASM would
not come back online after the reboot. It says it can't find the ASM disks,
but everything the sys admins look at say they are there and online.
What were the initial issues and why was a reboot the suggested fix?
We tried restoring the $GRID_HOME, but that didn't change anything. Our
Rimini support said a lot of libraries were missing from the $GRID_HOME, so
another DBA tried to getting the missing files. Still no luck. It's a
two-node RAC, but only one of the nodes was functional at the time we
rebooted the first node. The second has the same issues as the first, so we
can't bring anything online there either.
Can you follow the trail inside the alert/crs logs? Meaning, there might
be some clues as to what the underlying issue really and if it is missing
libraries or something else entirely.
Andy K
