昨天，一個朋友公司的數據庫崩潰。
這再次印證了我反復提到的一個命題：數據庫也需要休息。
每逢節假日，數據庫也經常會自我選擇放假。
以前我說：年終難終 進入數據庫事故多發期，一年一度今又是，記得另外一個聖誕節，我還和Biti一起在北京的時候，同樣遇到一個上海的朋友數據庫崩潰，我們遠程指導這位朋友恢復了數據。
這次的事情是這樣的。
1.首先主機宕機，磁盤出錯
看到以下這類錯誤，一般你的數據都很危險了
Dec 24 13:52:13 kernel: sda5: rw=0, want=18298437640, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=10384710304, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=8756273744, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=5023902272, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=6730428824, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=8884660792, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=9182513808, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=5002858800, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=6730428824, limit=163846872
Dec 24 13:52:13 kernel: attempt to access beyond end of device
Dec 24 13:52:13 kernel: sda5: rw=0, want=15872410168, limit=163846872
2.數據文件大量損壞
當然這次也不例外，大量文件損壞，dbv大量如下錯誤：
[oracle@stat datafile]$ dbv file=o1_mf_system_29448mn7_.dbf blocksize=8192
DBVERIFY: Release 10.2.0.2.0 - Production on Thu Dec 25 22:17:52 2008
Copyright (c) 1982, 2005, Oracle. All rights reserved.
DBVERIFY - Verification starting : FILE = o1_mf_system_29448mn7_.dbf
Page 40 is influx - most likely media corrupt
Corrupt block relative dba: 0x00400028 (file 1, block 40)
Fractured block found during dbv:
Data in bad block:
type: 6 format: 2 rdba: 0x00400028
last change scn: 0x0000.18990f0e seq: 0x1 flg: 0x06
spare1: 0x0 spare2: 0x0 spare3: 0x0
consistency value in tail: 0xbc120601
check value in block header: 0xc0cb
computed block checksum: 0xb003
Page 232 is influx - most likely media corrupt
Corrupt block relative dba: 0x004000e8 (file 1, block 232)
Fractured block found during dbv:
Data in bad block:
type: 6 format: 2 rdba: 0x004000e8
last change scn: 0x0000.18991b98 seq: 0x1 flg: 0x06
spare1: 0x0 spare2: 0x0 spare3: 0x0
consistency value in tail: 0x6c440601
check value in block header: 0x8d7f
computed block checksum: 0x77dc
3.控制文件損壞
啟動數據庫出現如下錯誤：
Wed Dec 24 17:08:52 2008
ALTER DATABASE MOUNT
Wed Dec 24 17:08:56 2008
Errors in file /opt/oracle/admin/stat/udump/stat_ora_4630.trc:
ORA-00600: internal error code, arguments: [kccpb_sanity_check_2], [11258908], [10375171], [0x0], [], [], [], []
Wed Dec 24 17:08:57 2008
ORA-600 signalled during: ALTER DATABASE MOUNT...
Wed Dec 24 17:09:01 2008
Starting ORACLE instance (normal)
Wed Dec 24 17:16:22 2008
Corrupt block 1 found during reading backup piece, file=/opt/oracle/product/db10g/dbs/snapcf_stat.f, corr_type=2
