MongoDB uses rs to implement ha and backup-1024programmer

The simplest deployment method of mongodb is single-node deployment. Before the National Day, I found an 8-core, 48G
The RAM server ran for a week under the pressure of 100+ concurrency, and no data loss or excessive server load was found

However, there are still some problems with single-node deployment. The first is that HA cannot be done. If the mongod is down, or the deployed server
If it is down, the application cannot work; second, it is not conducive to backup, because during backup, it will impose an additional burden on mongod, which may affect the business; third, it cannot do read-write separation. So in the production environment, still consider using cluster deployment

There are three cluster deployment methods supported by mongod:

1. master-slave

2. replica set

3. sharding

The master-slave can solve the backup problem, but it cannot perform HA transparently, so it is not good; sharding is a bright feature of mongo, which can automatically shard. However, according to my test results, sharding begins to show performance advantages after a single collection reaches the threshold of 20 million pieces of data, and sharding data is distributed, so backup will be more complicated and more servers are required. Bad for cost. So in the end I considered, or use replica
The cluster deployment in set mode is more suitable. It can solve the problems of HA, backup, and read-write separation

Basically adopt the TOPO recommended by the official website:

Ideally, it is best to deploy three mongod instances on separate servers, which requires three servers; for cost considerations, two secondary instances can also be deployed on the same server, because the primary has to deal with reading and writing Requests (reading and writing are not separated for the time being) require a lot of memory, and HA factors are considered, so it is better for the primary to ensure that it monopolizes one server, so a total of 2 servers are needed

1. Start mongod with -replset

You can also use the command line to start, but it is not conducive to management, so it is recommended to use the –config parameter to start, the configuration file is as follows:

port=2222
bind_ip=127.0.0.1
dbpath=/home/zhengshengdong/mongo_dbs/db1
fork=true
replSet=rs0
logpath=/home/zhengshengdong/mongo_log/log1.txt
logappend=true
journal=true

./mongod –config
/home/zhengshengdong/mongo_confs/mongo1.conf

Then follow the same pattern and start another 2 mongod instances

2. Initialize the replica set and add secondary

Use ./mongo –port 2222 to connect to the mongod instance to be used as the primary, and then execute the following commands in sequence

rs.initiate()
rs.conf()
rs.add(“host1:port”)
rs.add(“host2:port”)
3. Check

Execute on the primary instance

rs.status()
You should see something like the image below

Write the following code with java driver for verification

public static void main(String[] args) throws UnknownHostException {
ScheduledExecutorService executor = Executors
.newSingleThreadScheduledExecutor();
final MongoClient client = MongoConnectionHelper.getClient();
client.setWriteConcern(WriteConcern.REPLICA_ACKNOWLEDGED);
Runnable task = new Runnable() {
Private int i = 0;
@Override
Public void run() {
DB db = client.getDB(“yilos”);
DBCollection collection = db.getCollection(“test”);
&nboutputStream.java:92)
at
java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at org.bson.io.PoolOutputBuffer.pipe(PoolOutputBuffer.java:129)
at com.mongodb.OutMessage.pipe(OutMessage.java:236)
at com.mongodb.DBPort.go(DBPort.java:133)
at com.mongodb.DBPort.go(DBPort.java:106)
at com.mongodb.DBPort.findOne(DBPort.java:162)
at com.mongodb.DBPort.runCommand(DBPort.java:170)
at
com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:100)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:142)
… 16 more

It can be found that, in fact, this is not a mongo problem, but caused by a bug in this test code:

Runnable task = new Runnable() {
Private int i = 0;
@Override
Public void run() {
DB db = client.getDB(“yilos”);
DBCollection collection = db.getCollection(“test”);
DBObject o = new BasicDBObject(“name”, “MongoDB” + i).append(
“count”, i);
try {
Collection.insert(o);
} catch (Exception exc) {
exc.printStackTrace();
}
i++;
}
};
The problem lies here. The MongoException is caught in the try statement, but the code I wrote does not handle it. It simply prints out the exception, and then increments i to enter the next cycle.

It can also be judged from this that the java provided by mongo
The driver does not temporarily store the failed write operation in the queue and try again later; instead, it discards the write operation and throws an exception for the client to handle by itself. I think this design is okay, but the client must handle it. Since the follow-up is to use js
driver, so I will not continue to delve into this code, and study it carefully in the js driver. The key is to deal with mongo
exception, and set a higher level of write concern. The HA mechanism of the replica set itself is feasible

With 2 levels of backup:

The first layer is the natural backup provided by the cluster deployment, because there are 2 secondary
The node will always keep in sync with the primary, so at any time, the cluster has 2 complete mirror copies of the data

The second layer is to use the mongodump and fsync tools provided by mongo to obtain daily copies of key collections when the business load is low (3:00 a.m.) through manual execution or scripting. Backup collection is also performed on the secondary, which does not put additional pressure on the primary

Collected backups are saved locally or to other servers to avoid server storage damage, resulting in loss of all data and backups

At the same time, when starting mongod, open the journal parameter, so that in extreme cases (the above two backups fail), manual recovery can also be performed through the oplog

Collected backups are saved locally or to other servers to avoid server storage damage, resulting in loss of all data and backups

At the same time, when starting mongod, open the journal parameter, so that in extreme cases (the above two backups fail), manual recovery can also be performed through the oplog