It looks like there is no support for connecting to MongoDB with TLS enabled. When I run node wiki.js configure and I get to the part where it tests the connection to MongoDB, I get these results:
When MongoDB is configured with --sslMode requireSSL, I get "Error: write EPIPE" from Wiki.js. I confirmed the MongoDB server configuration is ok by connecting successfully with mongo --ssl <host>.
When MongoDB is configured with --sslMode preferSSL or --sslMode disabled, then Wiki.js is able to connect just fine and I can proceed with the configuration.
I haven't looked at the wiki.js code but at least the manual mentions TLS configuration settings for connecting to git, while the only setting mentioned for MongoDB is the URL. When TLS is added, there should also be a setting in Wiki.js to require SSL for the MongoDB connection, or to prefer SSL if it's available, or to only connect without SSL.