How MongoDB uses GridFS to save files
According to the official website,BSON objects in MongoDB are limited to 4MB in size. http://www.mongodb.org/display/DOCS/GridFS the So I re-wrote the operation class and used GridFS to save files. The code is very simple, but I have been in contact with it for a long time, and there is a problem that I can’t solve. I hope to generate a Guid by myself. _id is not the _id generated by mongodb, but it has been unable to solve it. I hope any expert will see it and give me some pointers, thank you! the This is written using the client provided by the Mongodb official website: the using System; using System.Collections.Generic; using System. Text; using System.IO; using MongoDB. Driver; using MongoDB. Driver. Builders; using MongoDB. Bson; using MongoDB.Driver.GridFS; namespace FileUtility { public class DocUtility { private string connStr = “”; public string ConnStr { get { If (string.IsNullOrEmpty(connStr)) { Throw new ArgumentNullException(“Connection string did not specify!”); } return connStr; } } public DocUtility() { ConnStr = System.Configuration.ConfigurationManager.AppSettings[“FileDb”]; } public DocUtility(string connectionString) { “ “ this. connStr = connectionString; } /// /// add a document to mongodb /// /// document bytes array /// the unique identity filename in mongodb Public string AddDoc(byte[]…
The difference between MongoDB and MySQL in use
Two databases that have developed very well in different fields, MongoDB (NoSQL) and MYSQL (relational), record their differences and the environments they are used in. MongoDB sacrifices space in exchange for strong scalability and a certain aspect of great speed improvement. . simply put. Relationship + transaction of relational database. Nosql headed by mongodb does not have It takes about 2 hours to insert 1 million data into a relational database, taking mysql as an example, and it takes 3 minutes to insert 1 million data into mongodb. . Insert 100w data, mongodb increased by 5G. The disk space of the mysql storage database has not increased significantly, not even 500M. Query speed, mongodb is very fast. But if you deal with the relationship, you will kneel down. For a simple example, a financial system is designed to delete a user (user foreign key to the user’s consumption record buy table). All the user’s buy records are deleted at the database layer at one time without any errors. But mongodb can’t do it, it doesn’t have foreign keys. We can only delete the user first, and then find out all the records belonging to this user in the buy and…
Performance test and comparison of two databases, MongoDB and MySQL
Purpose Test the read and write performance of mysql and mongodb illustrate 1. mongodb uses the latest version 2.4 2. Mysql uses 5.0.95, and the storage engine uses MYISAM (the measured read and write performance of INNODB is only a fraction of that of MYISAM) 3. The test data has only two fields, id: Integer (index field) increasing from 0 data: data field, tested the performance of data 1KB and 24KB respectively 4. When testing read performance, the query id is a random value in the inserted id 5. There are 50 concurrent read and write processes 6. Pressing machine: 4 cores/8G, DB machines were tested and deployed on ordinary machines (4 cores/8G memory) and high-end machines (24 cores/132G memory), and the hard disk is an ordinary mechanical hard disk 7. The pressing machine and the DB machine are directly connected through a 1000M network card 1. DB is deployed on a common machine 1. The data size is 1KB, and the number of reads and writes is 400W a) mongodb the Total Data Size QPS iowait net Read 11G 14600 <1% 20Mb/s Write 11G 10630 ~100% <17Mb/s b) mysql the Total Data Size QPS iowait net Read 4G…
What are the advantages of MongoDB over MySQL
The original title is “Is MongoDB the next big thing?”, please allow me to put big thing is equivalent to MySQL, the most widely used open source database. As one of the most popular NoSQL at the moment, let’s take a look at its specific advantages and disadvantages. Features of MongoDB Simple query statement, no Join operation Document-type storage, its data is stored in the binary Json format Bson. Its data is like Ruby hashes, or Python dictionaries, or PHP arrays Sharding, MongoDB provides auto-sharding to achieve data scalability GridFS, the file storage API provided by MongoDB Array index, you can index an array property in the document MapReduce, which can be used for complex statistics and parallel computing High performance, by using mmap and timing fsync, frequent IO is avoided, making it more performant Advantages of MongoDB High performance, very fast (if you have enough RAM) There is no fixed table structure, and there is no need for data migration to modify the table structure The query language is simple and easy to use Use Sharding to achieve horizontal expansion Easy to deploy When using MongoDB, you have to remember the following points: MongoDB assumes you have large disk…
Distributed file storage system based on MongoDB: GridFS
GridFS is a distributed file system on top of MongoDB. It utilizes MongoDB’s distributed storage mechanism and stores file data and file metadata through MongoDB. It has the advantages of both document database and file system. GridFS is the product of the current big data trend and complex data analysis needs. To put it simply, GridFS realizes the file system by storing file data and file metadata in MongoDB, and handles failover and data integration through replication (Replication). It can also be used for read expansion, hot backup or offline batch processing. The data source can automatically split data through sharding, realize big data storage and load balancing, realize lightweight file system interface through database management and query of documents in the collection (including MapReduce) and Search and Analytics. A basic idea of GridFS is that large files can be divided into many blocks, and each block is stored as a separate document, so large files can be stored. Since MongoDB supports storing binary data in documents, the storage overhead of blocks can be minimized. GridFS uses MongoDB’s replication, sharding and other mechanisms to implement distributed file storage, and uses MongoDB for management and complex analysis. GridFS uses two documents…
Introduction to the use of MongoDB and GridFS image storage
Usually use MySql, When Oracle does image storage, it directly uploads files to the server and stores them in a fixed directory on the hard disk, and only the path is stored in the table. Now use MongoDB, learn about MongoDB GridFS. MongoDB GridFS is a submodule of mongodb. Using GridFS can store files persistently based on mongodb. It also supports distributed applications (file distribution storage and reading). GridFS is a mongodb A tool for users to store large objects. For mongodb, BSON format data (document) storage has a size limit, the maximum is 16M. However, in actual system development, there is often a function of uploading pictures or files, and these files may be large in size ..We can use Gridfs to assist in the management of these files. The file table of Mongo GFS is composed of table name.files and table name.chunks Composition, the former is the composition of file information, the latter is the content of the file, and the two are associated with files_id through _id. GridFS will divide large file objects into multiple small chunks (file fragments), generally 256k/piece, and each chunk will be stored in the chunks collection as a document (document) of mongodb.…
How SpringData integrates MongoDB access
Spring Source groaning, from February 2011 to February 2011, finally released spring-data-mongo-1.0.1 to Release. Versions from 1.0.0.M1 to 1.0.0.M3 are called Spring Data Document. 1.0.0.M4 began to be renamed Spring Data MongoDB 1.0.0 M4, but the official website does not specify it. At first glance, it is a bit inexplicable, especially when MongoTemplate moves from org.springframework.data.document.mongodb to org.springframework.data.mongodb.core, the HelloWorldExample on the official website still uses org.springframework.data.document.mongodb as an example, which really causes a lot of misleading. Spring Data Mongo needs to rely on the Spring Framework, therefore, first you need to download the Spring Framework jar package, create a new Web project, and import the Spring Framework jar package. This article references the following components: Among them, in addition to several component packages of Spring, the Driver of MongoDB is also referenced: mongo-2.7.3.jar; component dwr.jar of DWR3.0; spring-data-mongdb-1.0.1.RELEASE.jar and Spring Data public component spring-data-commons-core-1.2.1.RELEASE.jar. It should be noted that the Spring used in this article The component version of Data Mongo needs to use Spring Framework 3.0.7 or above, otherwise an error will be reported when the program runs. the After introducing the above components, you need to modify the Spring configuration file ApplicationContext.xml file to introduce Spring Data…
How to update the MongoDB database in the program in Java
If the document has been stored, and now you want to add a description attribute to the document, you can update it like this: Mongo mongo = “bh_code_a_Java_keywords”>new Mongo( “bh_code_a_Java_keywords”>this.host, “bh_code_a_Java_keywords”>this.port); GridFS fs = “bh_code_a_Java_keywords”>new GridFS(mongo.getDB(“database”)); GridFSDBFile file = fs.find( “bh_code_a_Java_keywords”>new ObjectId(document.getId())); file.put( “bh_code_a_Java_string”>”description”, document.getDescription()); file. save(); You can also use the update method of DBCollection to update, DBObject queryCOndition= “bh_code_a_Java_keywords”>new BasicDBObject(); queryCondition. put( “bh_code_a_Java_string”>”_id”, document.getId()); Mongo mongo = “bh_code_a_Java_keywords”>new Mongo( “bh_code_a_Java_keywords”>this.host, “bh_code_a_Java_keywords”>this.port); DB db = mongo.getDB( “bh_code_a_Java_string”>”database”) DBCollection c = db.getCollection( “bh_code_a_Java_string”>”fs.files”); DBObject updatedValue= “bh_code_a_Java_keywords”>new BasicDBObject(); updatedValue. put( “bh_code_a_Java_string”>”description”, document.getDescription()); DBObject updateSetValue= “bh_code_a_Java_keywords”>new BasicDBObject( “bh_code_a_Java_string”>”$set”,updatedValue); c.update(updateSetValue, updateSetValue);
How to use MongoDB’s insert and update updates
When we use MongoDB’s update, if we set the third parameter to true, a new document will be created if no document is found. The following is an example: >db.foo.find() { “_id” : ObjectId(“4e9165cf717ed94f8289ac0c”), “bar” : “baz”} >db.foo.update({“bar”:”baz”}, {“$inc”:{“count”:3}}) >db.foo.find() { “_id” : ObjectId(“4e9165cf717ed94f8289ac0c”), “bar” : “baz”, “count” : 3 } >db.foo.update({“bar”:”bazz”}, {“$inc”:{“count”:3}}) >db.foo.find() { “_id” : ObjectId(“4e9165cf717ed94f8289ac0c”), “bar” : “baz”, “count” : 3 } > db.foo.update({“bar”:”bazz”}, {“$inc”:{“count”:3}},true) > db.foo.find() { “_id” : ObjectId(“4e9165cf717ed94f8289ac0c”), “bar” : “baz”, “count” : 3 } { “_id” : ObjectId(“4e916661739f1da5452a4dfe”), “bar” : “bazz”, “count” : 3 } If the third parameter is not set to true in update, no action will be taken when the document to be updated cannot be found; if the third parameter is set to true, a new document will be created if no document is found.
Summary of the advantages and disadvantages of MongoDB and MySQL
Compared with relational databases, the advantages of MongoDB: ①Weak consistency (eventually consistent), which can better guarantee the user’s access speed: For example, in a traditional relational database, a COUNT operation locks the data set so that the exact value in the “current” situation is guaranteed. This in some cases, such as For example, it is very important to check the account information through ATM, but for Wordnik, the data is constantly updated and growing. Late. What they need is an “approximately” number and faster processing speed. But in some cases MongoDB will lock the database. If there are hundreds of requests at this point, they can pile up and cause a lot of problems. We use the following optimization to avoid locking: Before each update, we will first query the records. Query operations place objects in memory, so updates are as fast as possible. In a master/slave deployment scenario, the slave nodes can be run with the “-pretouch” parameter, which also achieves the same effect. Use multiple mongod processes. We split the database into multiple processes based on access patterns. ②The storage method of the document structure can obtain data more conveniently. For a hierarchical data structure, if such data…
