We all know how critical it is to get our data back as soon as possible. To achieve this, as you all might be aware, we have two methods of restore available for MongoDB: logical and physical.
Of course, this depends on the type of backup we have configured. For large data sets, it is advisable to use physical backups which offer faster backup and restore times. I’ve used Percona Backup for MongoDB (PBM) to take a physical backup in this demo.
But here’s a catch. It’s very complicated to restore specific collection(s) from a physical backup, be it any kind of backup method like volume snapshot, cold rsync data file copies, or a Hotbackup/Percona Backup for MongoDB.
The simplest, or what almost everyone knows, is to restore the physical dump, a.k.a. data files, onto a temporary “mongod” and get the logical dump of that collection(s) manually first, followed by a logical restore to the original cluster. This is usually done with the conjunction of mongodump and mongorestore.
But there’s another way where we can avoid provisioning up a temporary mongod and then a generally slow logical dump, a.k.a. mongodump.
To achieve this, in this blog post we’ll be taking a look at the “wt” utility which will help us to perform a restore.
As a pre-requisite, we need to first build “wt” from either their GitHub repo or download a tar archive file and build it to a specific version. Now let’s jump straight into the steps now.
In this blog, we’ll be building using the GitHub repo.
1. Clone the WiredTiger repo
$ git clone https://github.com/wiredtiger/wiredtiger.git
$ cd wiredtiger
2. Check your mongod version and checkout the same branch version.
$ git branch --all | grep mongodb
$ git checkout remotes/origin/mongodb-5.0 (I was running version 5.0)
3. Depending on the type of compression method we’re using, we will install its library and configure its WiredTiger library extension. Mostly snappy is being used currently so I’ve used the same method. But depending upon what compression method you have, you can provide the same library path after installing/configuring it.
$ ./configure --enable-zstd --enable-zlib --enable-lz4 -enable-snappy (if it fails with 'configure: error', then install that library first</strong>)
$ yum install libzstd-devel zlib-devel lz4-devel snappy-devel
$ make -j $(nproc)
Once we have the necessary dependencies installed and verified, we can execute the commands below to dive straight into the action of restoring the dropped collection.
Restoration time
1. Find out the relevant URI of the collection you want to restore. This can be checked on the original cluster:
> demo:PRIMARY> db.demo.stats().wiredTiger.uri
statistics:table:collection-36-8063732008498576985
> demo:PRIMARY> db.adminCommand({'getCmdLineOpts':1}).parsed.storage.dbPath
/var/lib/mongo
2. It’s necessary to check the compression method of the collection. Accordingly, we need to use the respective WiredTiger library extension. Look for ‘block_compressor’:
> demo:PRIMARY> db.demo.stats().wiredTiger.creationString
.........block_compressor=snappy......... //truncated output
3. There are different ways to take a physical dump like Percona Backup for MongoDB, Hotbackup, Snapshot based, or even a rsync to a separate volume/path. Let’s now take a physical backup using Percona Backup for MongoDB and then drop our collection.
========
FS /backup
Snapshots:
2022-10-26T03:05:19Z 10.87MB <physical> [restore_to_time: 2022-10-26T03:05:21Z]
$ pbm describe-backup 2022-10-26T03:05:19Z
name: "2022-10-26T03:05:19Z"
opid: 6358a3ef2722ffe26150f98b
type: physical
last_write_time: "2022-10-26T03:05:21Z"
last_transition_time: "2022-10-26T03:05:26Z"
mongodb_version: 4.4.16-16
pbm_version: 2.0.1
status: done
size_h: 10.9 MiB
replsets:
- name: demo
status: done
last_write_time: "2022-10-26T03:05:21Z"
last_transition_time: "2022-10-26T03:05:25Z"
security: {}
$ ls -lrth /backup
total 12K
drwxr-xr-x. 3 mongod mongod 17 Oct 26 03:05 2022-10-26T03:05:19Z
-rw-r--r--. 1 mongod mongod 11K Oct 26 03:05 2022-10-26T03:05:19Z.pbm.json
$ ls -lrth /backup/2022-10-26T03:05:19Z/demo
total 28K
drwxr-xr-x. 4 mongod mongod 37 Oct 26 03:05 admin
drwxr-xr-x. 4 mongod mongod 37 Oct 26 03:05 local
-rw-r--r--. 1 mongod mongod 6.7K Oct 26 03:05 WiredTiger.backup.s2
drwxr-xr-x. 4 mongod mongod 37 Oct 26 03:05 config
-rw-r--r--. 1 mongod mongod 4.7K Oct 26 03:05 _mdb_catalog.wt.s2
-rw-r--r--. 1 mongod mongod 2.0K Oct 26 03:05 WiredTigerHS.wt.s2
drwxr-xr-x. 4 mongod mongod 37 Oct 26 03:05 percona
-rw-r--r--. 1 mongod mongod 1.9K Oct 26 03:05 sizeStorer.wt.s2
-rw-r--r--. 1 mongod mongod 68 Oct 26 03:05 WiredTiger.s2z
drwxr-xr-x. 2 mongod mongod 76 Oct 26 03:05 journal
As you can clearly see, using PBM we can list all our underlying data files which we’ll use for this demo purpose.
> demo:PRIMARY> use percona
switched to db percona
> demo:PRIMARY> show collections
demo
> demo:PRIMARY> db.demo.countDocuments({})
1000000
> demo:PRIMARY>
> demo:PRIMARY> db.demo.drop() //accidentally dropped collection</strong>
true
> demo:PRIMARY> show collections
> demo:PRIMARY> db
percona
4. Take the “wt dump” of the URI file from Step 1 into a mongodb known bson format. A regular output of “wt dump” would be in binary hex string format, otherwise.
$ ./wt -v -h /backup -C "extensions=[/root/wiredtiger/ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" dump -x file:collection-36-8063732008498576985.wt | tail -n +7 | awk 'NR%2 == 0 { print }' | xxd -r -p > /backup/percona.demo.bson
If you have got a different compression method for the collection or on a global level, you can use its respective WiredTiger library extension.
/clonedPath/wiredtiger/ext/compressors/<compressionMethod>/.libs/libwiredtiger_<compMethod>.so
Note: Don’t forget to append ‘.wt’ at the end of URI and add the prefix “table:” or “file:”, as within WiredTiger, all collection files are in the table or file format.
Let’s look at and understand the different flags used in the above command.
- Extensions: based on our compression method, we have used respective compression WiredTiger library extension
- [ -v, -x , dump, -h, -C]: These are WiredTiger binary flags to take the dump from a URI file in hex raw string in verbose style with default or command line provided configuration using “-C”
- Tail is only used to trim the top seven header lines from the output of “wt dump”
- awk is used to filter out just the keys (line number NR%2 == 1) or values (line number NR%2 == 0) in the pretty-print or hex mode of wt dump.
- xxd with conjunction of ‘-r’ and ‘-p’ is to convert raw hex strings into mongodb known bson format
- Finally, we’re redirecting the output to filename “percona.demo.bson”. This is very important to keep the output filename as per WiredTiger catalog ident which is nothing but a proper full namespace of collection URI. In our case, it was “percona.demo”.
- If needed, it can be validated using the below command:
$ ./wt -v -h /backup -C "extensions=[/root/wiredtiger/ext/compressors/snappy/.libs/libwiredtiger_snappy.so]" dump -x table:_mdb_catalog | tail -n +7 | awk 'NR%2 == 0 { print }' | xxd -r -p | bsondump --quiet | jq -r 'select(. | has("md")) | [.ident, .ns] | @tsv' | sort | grep percona | awk '{print $2}'
percona.demo
5. Finally, restore bson using the native mongorestore.
$ mongorestore -authenticationDatabase "admin" --port 37017 -d percona -c demo /backup/percona.demo.bson
2022-10-17T03:03:43.232+0000 checking for collection data in /backup/percona.demo.bson
2022-10-17T03:03:43.240+0000 restoring percona.demo from /backup/percona.demo.bson
2022-10-17T03:03:46.231+0000 [####################....] percona.demo 106MB/128MB (83.4%)
2022-10-17T03:03:46.848+0000 [########################] percona.demo 128MB/128MB (100.0%)
2022-10-17T03:03:46.848+0000 finished restoring percona.demo (1000000 documents, 0 failures)
2022-10-17T03:03:46.848+0000 1000000 document(s) restored successfully. 0 document(s) failed to restore.
$ mongo --quiet --port 37017 percona
demo:PRIMARY> show collections
demo
demo:PRIMARY> db.demo.countDocuments({})
1000000
demo:PRIMARY> db.demo.findOne()
{
"_id" : ObjectId("634cba61d42256b5fb5c9033"),
"name" : "Blanche Potter",
"age" : 38,
"emails" : [
"filvo@av.hm",
"pamjiiw@ewdofmik.sh",
"opaibizaj@ha.ch"
]
}
demo:PRIMARY>
SUMMARY
A few things to consider
1. You all must be thinking, what about indexes, right? 
The same process can be done by dumping the index*.wt files. But it’s rather complex as dumped keys and values have slightly different formats. I’ll soon cover it in a separate blog. Also interesting to mention is that WiredTiger maintains multiple index* URI files for every index, thus it’s better to build manually with the “createIndex” command which is a far easier approach.
demo:PRIMARY> db.demo.getIndexKeys()
[ { "_id" : 1 }, { "a" : 1 }, { "b" : 1 } ]
demo:PRIMARY> db.demo.stats({'indexDetails': true}).indexDetails['_id_'].uri
statistics:table:percona/index/35-2625234990440311433
demo:PRIMARY> db.demo.stats({'indexDetails': true}).indexDetails['a_1'].uri
statistics:table:percona/index/69-2625234990440311433
demo:PRIMARY> db.demo.stats({'indexDetails': true}).indexDetails['b_1'].uri
statistics:table:percona/index/71-2625234990440311433
2. In order to perform point-in-time recovery, incremental oplogs still need to be replayed on top of the restored backup, which we have covered.
3. This method is applicable for sharded clusters as well (both unsharded and sharded collections) but there are a few additional steps that need to be taken. We’ll cover similarly detailed demos in upcoming blogs.
Before I wrap up, let’s talk about some drawbacks since this is a bit risky and complicated approach. Hence test it out first in your lab or test environment first to get familiarized with WiredTiger internals before jumping straight into production.
Cons
- Overall it’s a bit complicated approach as one has to have a clear understanding of “wt” utility and its internals.
- Doesn’t restore indexes and needs to be built separately, as mentioned.
Percona Distribution for MongoDB is a freely available MongoDB database alternative, giving you a single solution that combines the best and most important enterprise components from the open source community, designed and tested to work together.
To follow up on my previous Kubernetes articles:
Percona is a leading provider of unbiased open source database solutions that allow organizations to easily, securely, and affordably maintain business agility, minimize risks, and stay competitive.
Not long ago, we found a long-running query in one of our clients on 
This week is 
In this blog post, we will discuss delayed members of a replica set and how it can be useful. A delayed member of the replica set is similar to like other members of the replica set but its data set mirrors the set delayed state of the data. Let’s say if a delayed node is delayed by one hour and the current time is 11:00, then its data set reflects recent operation till 10:00.