C#使用BerkeleyDB操作簡介

Berkeley DB是歷史悠久的嵌入式數據庫系統，主要應用在UNIX/LINUX操作系統上。Berkeley DB的存儲的是key/value鍵值對，可以理解為硬盤上的超級hash表。其可以管理256TB數據，而且能支撐幾千個並發訪問。目前Berkeley DB有C++版和Java版。所以，我們需要一個訪問的中間轉換，已經有人發布了C#的API。可以從 Berkeley DB for .NET 上面找到，現在最新版是0.95版本，可以支持4.3和4.5版。本篇將以4.5版做實例。BerkeleyDB的版本可以在http://www.oracle.com/technology/products/berkeley-db/index.html下載，當前最新版本為4.7版。4.5 C++版的Berkeley DB可以在http://www.oracle.com/technology/software/products/berkeley-db/db/index.html這裡下載。

By Birdshover@ 博客園 http://www.cnblogs.com/birdshover/

下載到Berkeley DB for .Net的API——libdb-dotnet_0_95.zip後，就可以開始使用了。首先在libdb-dotnet_0_95.zip解壓縮的bin目錄找到libdb_dotNET45.dll，這個就是4.5版本使用的dll。新建項目，引用這個dll。注意，自己編譯源碼可能會編譯不過，主要是因為裡面一些委托和委托的參數可見性不一致造成的。把那些參數用到的class 或者struct都調成public即可。

BerkeleyDB的數據庫操作需要借助DbBTree類。因此需要先得到DbBTree的實例，但是DbBTree類會對其它幾個類有依賴，必須依賴其它幾個類才能創建。

下面代碼就是初始化得到DbBTree實例的一個過程。

/// <summary>
/// 數據庫目錄
/// </summary>
private string directory;
/// <summary>
/// 數據庫文件名
/// </summary>
private string dbName;

private DbBTree btree;
private Txn txn;
private Db db;
private Env env;
/// <summary>
/// 初始化
/// </summary>
private void Init()
{
env = new Env(EnvCreateFlags.None);
Env.OpenFlags envFlags =
Env.OpenFlags.Create |
Env.OpenFlags.InitLock |
Env.OpenFlags.InitLog |
Env.OpenFlags.InitMPool |
Env.OpenFlags.InitTxn |
Env.OpenFlags.Recover;
env.Open(directory, envFlags, 0);
txn = env.TxnBegin(null, Txn.BeginFlags.None);
db = env.CreateDatabase(DbCreateFlags.None);
btree = (DbBTree)db.Open(txn, dbName, null, DbType.BTree, Db.OpenFlags.Create, 0);
}

另外Berkeley DB數據庫的操作需要借助於序列化。

/// <summary>
/// 二進制序列化
/// </summary>
private BinaryFormatter formatter;
/// <summary>
/// 鍵內存流
/// </summary>
private MemoryStream keyStream;
/// <summary>
/// 內容內存流
/// </summary>
private MemoryStream dataStream;

private void StreamInit()
{
formatter = new BinaryFormatter();
keyStream = new MemoryStream();
dataStream = new MemoryStream();
}

Berkeley DB是鍵值數據庫，因此定義一個獲取鍵接口：

public interface IPut
{
string Key { get; }
}

一、數據庫的保存與更新

public bool Set(IPut put)
{
Reset();

keyStream.Position = 0;
formatter.Serialize(keyStream, put.Key);
DbEntry key = DbEntry.InOut(keyStream.GetBuffer(), 0, (int)keyStream.Position);
dataStream.Position = 0;
formatter.Serialize(dataStream, put);
DbEntry data = DbEntry.InOut(dataStream.GetBuffer(), 0, (int)dataStream.Position);
WriteStatus status = btree.Put(txn, ref key, ref data);
switch (status)
{
case WriteStatus.Success:
return true;
case WriteStatus.NotFound:
case WriteStatus.KeyExist:
default:
return false;
}
}

上述代碼就可以保存鍵值。顯示對鍵值進行序列化，然後再保存。保存完有三個狀態，可以一一處理。

二、數據庫的刪除

刪除最為簡單

public bool Remove(IPut put)
{
keyStream.Position = 0;
formatter.Serialize(keyStream, put.Key);
DbEntry key = DbEntry.InOut(keyStream.GetBuffer(), 0, (int)keyStream.Position);
DeleteStatus status = btree.Delete(txn, ref key);
switch (status)
{
case DeleteStatus.NotFound:
case DeleteStatus.Success:
return true;
case DeleteStatus.KeyEmpty:
default:
return false;
}
}

三、關於添加和刪除

添加和刪除並沒有真正得進行添加和刪除。必須執行Commit操作：

private bool iscomit = false;
public void Commit()
{
txn.Commit(Txn.CommitMode.None);
iscomit = true;
}
　四、尋找鍵

用鍵查詢值，和hash表一樣使用。

public bool Get(ref IPut put)
{
keyStream.Position = 0;
formatter.Serialize(keyStream, put.Key);
DbEntry key = DbEntry.InOut(keyStream.GetBuffer(), 0, (int)keyStream.Position);
dataStream.SetLength(dataStream.Capacity);
DbEntry data = DbEntry.Out(dataStream.GetBuffer());

while (true)
{
ReadStatus status = btree.Get(txn, ref key, ref data, DbFile.ReadFlags.None);

switch (status)
{
case ReadStatus.Success:
dataStream.Position = 0;
dataStream.SetLength(data.Size);
put = (IPut)formatter.Deserialize(dataStream);
return true;
case ReadStatus.BufferSmall: //擴容
if (key.Buffer.Length < key.Size)
{
keyStream.SetLength(key.Size);
key = DbEntry.Out(keyStream.GetBuffer());
}
if (data.Buffer.Length < data.Size)
{
dataStream.SetLength(data.Size);
data = DbEntry.Out(dataStream.GetBuffer());
}
continue;
case ReadStatus.NotFound:
case ReadStatus.KeyEmpty:
default:
return false;
}
}
}

五、遍歷

public List<IPut> Find()
{
List<IPut> custList = new List<IPut>();
using (DbBTreeCursor cursor = btree.OpenCursor(txn, DbFileCursor.CreateFlags.None))
{
IPut cust = null;
while (GetNextRecord(cursor, ref cust))
custList.Add(cust);
}
return custList;
}

private bool GetNextRecord(DbBTreeCursor cursor, ref IPut cust)
{
ReadStatus status;
keyStream.SetLength(keyStream.Capacity);
dataStream.SetLength(dataStream.Capacity);
DbEntry key = DbEntry.Out(keyStream.GetBuffer());
DbEntry data = DbEntry.Out(dataStream.GetBuffer());
do
{
status = cursor.Get(ref key, ref data, DbFileCursor.GetMode.Next, DbFileCursor.ReadFlags.None);
switch (status)
{
case ReadStatus.NotFound: return false;
case ReadStatus.KeyEmpty: continue; // skip deleted records
case ReadStatus.BufferSmall:
if (key.Buffer.Length < key.Size)
{
keyStream.SetLength(key.Size);
key = DbEntry.Out(keyStream.GetBuffer());
}
if (data.Buffer.Length < data.Size)
{
dataStream.SetLength(data.Size);
data = DbEntry.Out(dataStream.GetBuffer());
}
continue;
case ReadStatus.Success:
dataStream.Position = 0;
dataStream.SetLength(data.Size);
cust = (IPut)formatter.Deserialize(dataStream);
return true;
default:
return false;
}
} while (true);
}

六、完整操作封裝

public interface IPut
{
string Key { get; }
}

public class BDBManager : IDisposable
{
/// <summary>
/// 數據庫目錄
/// </summary>
private string directory;
/// <summary>
/// 數據庫文件名
/// </summary>
private string dbName;

private DbBTree btree;
private Txn txn;
private Db db;
private Env env;

/// <summary>
/// 二進制序列化
/// </summary>
private BinaryFormatter formatter;
/// <summary>
/// 鍵內存流
/// </summary>
private MemoryStream keyStream;
/// <summary>
/// 內容內存流
/// </summary>
private MemoryStream dataStream;


public BDBManager(string directory, string dbName)
{
this.directory = directory;
this.dbName = dbName;

Init();
StreamInit();
}

public bool Set(IPut put)
{
Reset();

keyStream.Position = 0;
formatter.Serialize(keyStream, put.Key);
DbEntry key = DbEntry.InOut(keyStream.GetBuffer(), 0, (int)keyStream.Position);
dataStream.Position = 0;
formatter.Serialize(dataStream, put);
DbEntry data = DbEntry.InOut(dataStream.GetBuffer(), 0, (int)dataStream.Position);
WriteStatus status = btree.Put(txn, ref key, ref data);
switch (status)
{
case WriteStatus.Success:
return true;
case WriteStatus.NotFound:
case WriteStatus.KeyExist:
default:
return false;
}
}

private bool iscomit = false;
public void Commit()
{
txn.Commit(Txn.CommitMode.None);
iscomit = true;
}

public List<IPut> Find()
{
List<IPut> custList = new List<IPut>();
using (DbBTreeCursor cursor = btree.OpenCursor(txn, DbFileCursor.CreateFlags.None))
{
IPut cust = null;
while (GetNextRecord(cursor, ref cust))
custList.Add(cust);
}
return custList;
}

public bool Get(ref IPut put)
{
keyStream.Position = 0;
formatter.Serialize(keyStream, put.Key);
DbEntry key = DbEntry.InOut(keyStream.GetBuffer(), 0, (int)keyStream.Position);
dataStream.SetLength(dataStream.Capacity);
DbEntry data = DbEntry.Out(dataStream.GetBuffer());

while (true)
{
ReadStatus status = btree.Get(txn, ref key, ref data, DbFile.ReadFlags.None);

switch (status)
{
case ReadStatus.Success:
dataStream.Position = 0;
dataStream.SetLength(data.Size);
put = (IPut)formatter.Deserialize(dataStream);
return true;
case ReadStatus.BufferSmall: //擴容
if (key.Buffer.Length < key.Size)
{
keyStream.SetLength(key.Size);
key = DbEntry.Out(keyStream.GetBuffer());
}
if (data.Buffer.Length < data.Size)
{
dataStream.SetLength(data.Size);
data = DbEntry.Out(dataStream.GetBuffer());
}
continue;
case ReadStatus.NotFound:
case ReadStatus.KeyEmpty:
default:
return false;
}
}
}

public bool Remove(IPut put)
{
Reset();

keyStream.Position = 0;
formatter.Serialize(keyStream, put.Key);
DbEntry key = DbEntry.InOut(keyStream.GetBuffer(), 0, (int)keyStream.Position);
DeleteStatus status = btree.Delete(txn, ref key);
switch (status)
{
case DeleteStatus.NotFound:
case DeleteStatus.Success:
return true;
case DeleteStatus.KeyEmpty:
default:
return false;
}
}

public void Dispose()
{
if (!iscomit)
Commit();
db.Close();
db.Close();
}

private void Reset()
{
iscomit = false;
}

private void Init()
{
env = new Env(EnvCreateFlags.None);
Env.OpenFlags envFlags =
Env.OpenFlags.Create |
Env.OpenFlags.InitLock |
Env.OpenFlags.InitLog |
Env.OpenFlags.InitMPool |
Env.OpenFlags.InitTxn |
Env.OpenFlags.Recover;
env.Open(directory, envFlags, 0);
txn = env.TxnBegin(null, Txn.BeginFlags.None);
db = env.CreateDatabase(DbCreateFlags.None);
btree = (DbBTree)db.Open(txn, dbName, null, DbType.BTree, Db.OpenFlags.Create, 0);
}

private void StreamInit()
{
formatter = new BinaryFormatter();
keyStream = new MemoryStream();
dataStream = new MemoryStream();
}

private bool GetNextRecord(DbBTreeCursor cursor, ref IPut cust)
{
ReadStatus status;
keyStream.SetLength(keyStream.Capacity);
dataStream.SetLength(dataStream.Capacity);
DbEntry key = DbEntry.Out(keyStream.GetBuffer());
DbEntry data = DbEntry.Out(dataStream.GetBuffer());
do
{
status = cursor.Get(ref key, ref data, DbFileCursor.GetMode.Next, DbFileCursor.ReadFlags.None);
switch (status)
{
case ReadStatus.NotFound: return false;
case ReadStatus.KeyEmpty: continue; // skip deleted records
case ReadStatus.BufferSmall:
if (key.Buffer.Length < key.Size)
{
keyStream.SetLength(key.Size);
key = DbEntry.Out(keyStream.GetBuffer());
}
if (data.Buffer.Length < data.Size)
{
dataStream.SetLength(data.Size);
data = DbEntry.Out(dataStream.GetBuffer());
}
continue;
case ReadStatus.Success:
dataStream.Position = 0;
dataStream.SetLength(data.Size);
cust = (IPut)formatter.Deserialize(dataStream);
return true;
default:
return false;
}
} while (true);
}
}

調用方法：

首先要有一個寫入的實體類，必須可以序列化,並且實現IPut接口：

[Serializable()]
class Item : IPut
{
public string Name { get; set; }
public string Text { get; set; }
public int ID { get; set; }

public override string ToString()
{
return string.Format("ID:{0} Key:{1}", ID, Name);
}

public string Key
{
get { return Name; }
}
}
　操作：

using (BDBManager manager = new BDBManager("db", "db.dat"))
{
bool success = manager.Set(new Item() { ID = 1000, Name = "Test",Text = "213" });
Console.WriteLine(string.Format("set is {0}", success));
}

using (BDBManager manager = new BDBManager("db", "db.dat"))
{
IPut put = new Item() { Name = "Test" };
bool success = manager.Get(ref put);
Console.WriteLine(string.Format("read is {0},item : {1}", success, put.ToString()));
}

using (BDBManager manager = new BDBManager("db", "db.dat"))
{
IPut put = new Item() { Name = "Test" };
bool success = manager.Remove(put);
Console.WriteLine(string.Format("remove is {0},item : {1}", success, put.ToString()));
}

using (BDBManager manager = new BDBManager("db", "db.dat"))
{
List<IPut> list = manager.Find();
foreach (var item in list)
{
Console.WriteLine(item.ToString());
}
}
Console.WriteLine("end");
Console.ReadKey();