国产探花免费观看_亚洲丰满少妇自慰呻吟_97日韩有码在线_资源在线日韩欧美_一区二区精品毛片,辰东完美世界有声小说,欢乐颂第一季,yy玄幻小说排行榜完本

首頁 > 開發 > 綜合 > 正文

MongoDB的分布式文件存儲系統

2024-07-21 02:53:10
字體:
來源:轉載
供稿:網友

概述

對于MongoDB的存儲基本單元BSON文檔對象,字段值可以是二進制類型,基于此特點,我們可以直接在MongoDB中存儲文件,但是有一個限制,由于MongoDB中單個BSON對象不能大于16MB,故而如果需要存儲更大的文件,就需要GridFS了。

小文件存儲系統與GridFS文件存儲

我們先看下MongoDB存儲小文件系統的例子:

先采用MongoDB的mongofiles執行文件上傳:

D:/MongoDB/Server/3.2/bin>mongofiles.exe list2017-03-06T13:41:03.283+0800 connected to: localhostD:/MongoDB/Server/3.2/bin>mongofiles.exe put E:/deliveryTask.doc2017-03-06T13:41:23.535+0800 connected to: localhostadded file: E:/deliveryTask.docD:/MongoDB/Server/3.2/bin>mongofiles.exe list2017-03-06T13:41:30.114+0800 connected to: localhostE:/deliveryTask.doc 2971

通過mongos命令查看文件存儲情況:

> use testswitched to db test> show collectionsfs.chunksfs.filesrestaurantsuser> db.fs.files.find(){ "_id" : ObjectId("58bcf683afa0fa20bc854a2b"), "chunkSize" : 261120, "uploadDate" : ISODate("2017-03-06T05:41:23.604Z"), "length" : 2971, "md5" : "5434b803306299fff57c8a54d3adf78b", "filename" : "E://deliveryTask.doc" }

可以看到文件上傳成功了

由于本章節主要涉及到的是部分理論以及運維實踐,暫不涉及具體代碼的開發實現(具體代碼將以java為例在后面章節中介紹)。

上傳一個大于16MB的文件試一試:

D:/MongoDB/Server/3.2/bin>mongofiles.exe put E:/synch.rar2017-03-06T14:33:11.028+0800 connected to: localhostadded file: E:/synch.rarD:/MongoDB/Server/3.2/bin>mongofiles.exe list2017-03-06T14:33:15.265+0800 connected to: localhostE:/deliveryTask.doc 2971E:/synch.rar 24183487

通過mongos命令查看文件存儲情況:

> db.fs.files.find(){ "_id" : ObjectId("58bcf683afa0fa20bc854a2b"), "chunkSize" : 261120, "uploadDate" : ISODate("2017-03-06T05:41:23.604Z"), "length" : 2971, "md5" : "5434b803306299fff57c8a54d3adf78b", "filename" : "E://deliveryTask.doc" }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "chunkSize" : 261120, "uploadDate" : ISODate("2017-03-06T06:33:12.013Z"), "length" : 24183487, "md5" : "bbfe4d8579372aa0729726185997e908", "filename" : "E://synch.rar" }

也成功了,

查看chunks:

> db.fs.chunks.find({},{data:0}){ "_id" : ObjectId("58bcf683afa0fa20bc854a2c"), "files_id" : ObjectId("58bcf683afa0fa20bc854a2b"), "n" : 0 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b2d"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 0 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b2e"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 1 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b2f"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 2 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b30"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 3 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b31"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 4 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b32"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 5 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b33"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 6 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b34"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 7 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b35"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 8 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b36"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 9 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b37"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 10 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b38"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 11 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b39"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 12 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b3a"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 13 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b3c"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 15 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b3b"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 14 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b3e"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 17 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b3d"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 16 }{ "_id" : ObjectId("58bd02a7afa0fa21d4a14b3f"), "files_id" : ObjectId("58bd02a7afa0fa21d4a14b2c"), "n" : 18 }Type "it" for more

可以看到大文件被分成了好多個chunk,那么為什么上傳16MB以上的文件也會成功呢,因為我們采用的就是GridFS系統存儲的文件,因為我們采用的是mongfiles方式上傳的文件。

下面分別是查詢、下載、刪除操作:

D:/MongoDB/Server/3.2/bin>mongofiles.exe search rar2017-03-06T14:45:31.974+0800 connected to: localhostE:/synch.rar 24183487D:/MongoDB/Server/3.2/bin>mongofiles.exe --local D:/mongodb_download.rar get E:/synch.rar2017-03-06T14:47:17.841+0800 connected to: localhostfinished writing to D:/mongodb_download.rarD:/MongoDB/Server/3.2/bin>mongofiles.exe delete E:/synch.rar2017-03-06T14:47:56.649+0800 connected to: localhostsuccessfully deleted all instances of 'E:/synch.rar' from GridFSD:/MongoDB/Server/3.2/bin>mongofiles.exe list2017-03-06T14:48:03.886+0800 connected to: localhostE:/deliveryTask.doc 2971

實際上,我們還可以自定義集合的前綴,默認是fs,或者設置chunk的大小,默認是256KB。

總結

那么在實際場景的分布式文件存儲系統中如何確定改用哪種存儲方案呢,可以采用如下方式: 1. 對于用戶上傳的任何文件,在客戶端進行大小判斷; 2. 當文件大小小于16MB時,則直接存儲到MOngoDB普通集合中 3. 當文件大小大于16MB時,上傳到GridFS中,利用集合fs.files以及fs.chunks來保存文件 4. 當用戶下載文件時,再根據不同文件的大小屬性不同到不同的集合中查找

另外,對于fs.chunks文件我們可以分片存儲,片鍵可以選擇索引字段{“files_id”},該字段盡量保證了此文件在所有被分割的chunk都在同一個片上,fs.files不需要分片,此集合只保存文件的元數據信息,數據量不大,同時還可以設置默認塊大小(256KB)

需要注意的是:GridFS并不適合小文件存儲,因為從GridFS中讀取文件涉及到兩次查詢操作,先查詢fs.files集合,再查詢fs.chunks集合,chunks合并后獲取整個文件。

需要注意的另一點是:文件分塊大小為256KB,而分片的塊大小默認64MB,不要搞混了。


發表評論 共有條評論
用戶名: 密碼:
驗證碼: 匿名發表
主站蜘蛛池模板: 崇文区| 青龙| 喀喇| 淳安县| 仁布县| 日喀则市| 天长市| 乳源| 武邑县| 广州市| 扎鲁特旗| 金溪县| 郴州市| 辰溪县| 曲松县| 溧水县| 屏东县| 盐亭县| 武城县| 东港市| 徐汇区| 周至县| 贵溪市| 连南| 沛县| 洞头县| 图们市| 色达县| 兴安盟| 阜城县| 平罗县| 隆回县| 延吉市| 格尔木市| 百色市| 太湖县| 宁阳县| 新密市| 洪雅县| 呈贡县| 阳高县|