On 14.03.2013 22:07, Jeremy Schneider wrote: > Hey all - > > I'm writing a paper about top causes of downtime. As one component > of research, I'd like to get some input from you! > > One minute, two sentences. First sentence: describe what went down. > Second sentence: describe why. (I have to categorize all of these.) > Everyone should have at least one downtime story so I'm hoping for a > lot of feedback! > > Answer about any technology - database, operating system, etc. > > Thanks! > > -Jeremy > > During a big, big project developers need many test environments with 0 time restore on quite big db. Instead of backup etc. there were snapshots done, then on a snapshot RO there was a pool of storage for RW. Worked quite fine. But the storage array had a feature not too well known. which we become aware afterwards ;-) - if it lacked in space, it deleted the oldest snapshot. There were actually only those snapshots for this project - the rest environments had the space allocated explicitly. Somebody run copying quite large pack to this environment based on snapshot (or may be by mistake it was even copied the same RO snapshot mounted somewehere else). Without the feature RW space would run short ending the procedure with failure, but with it the RO snapshot was the oldest one and went to trash - no RO, no copy. As we asked the storage company about this strange behaviour, we have just got well known - "this is not a bug, ...' -- Pole na kazi ---------------------------------------------------------------------- Remigiusz Sokolowski <remigiusz.sokolowski@xxxxxxxxxx> pos : Senior DBA at DIiUSI addr : Nordea Bank Polska SA, Luzycka 6A st, 81-537 Gdynia, Poland phone : +48 58 667 17 43 mobile: +48 602 42 42 77 Nordea Bank Polska S.A. z siedziba w Gdyni, ul. Kielecka 2, 81-303 Gdynia, wpisana do Rejestru Przedsiebiorców Krajowego Rejestru Sadowego pod numerem: 0000021828, dla której dokumentacje przechowuje Sad Rejonowy Gdansk - Pólnoc w Gdansku, VIII Wydzial Gospodarczy Krajowego Rejestru Sadowego, o kapitale zakladowym i wplaconym w wysokosci: 277.493.500,00 zlotych, NIP: 586-000-78-20, REGON: 190024711-- //www.freelists.org/webpage/oracle-l