MongoDB history lessons never taught
An insightful article, exploring technical perspectives for widespread MongoDB adoption.
Three historical reasons why MongoDB is where it is now.
1. MongoDB was born out of necessity. There was a need for an app-centric solution and existing tabular databases required too much work.
MongoDB was founded in 2007 by Dwight Merriman, Eliot Horowitz, and Kevin Ryan — the team behind DoubleClick.
At the Internet advertising company DoubleClick (acquired by Google at $3.1 Billion in 2008 and later merged into Google Marketing Platform), the team developed and used many custom data stores to work around the shortcomings of existing databases. The business served 400,000 ads per second but often struggled with both scalability and agility. Frustrated, the team was inspired to create a database that tackled the challenges it faced at DoubleClick.
This was when MongoDB was born.
“MongoDB was born out of our frustration using tabular databases in large, complex production deployments. We set out to build a database that we would want to use, so that whenever developers wanted to build an application, they could focus on the application, not on working around the database.”
- Eliot Horowitz, CTO & Co-Founder MongoDB (2007–2020)
2. Rise of MEAN/MERN, JavaScript tooling ecosystem, and JSON. Just one language to do complete Full-stack development.
I remember a discussion with my flatmate Manish back in my college days around 2012-
Myself — “Bhai, What do I need to learn to create my own web application?”
Manish — “ I recommend going with LAMP stack, assuming you know HTML, CSS, and JS for frontend, you'll just need to learn SQL and PHP. You would also have to study a bit about configuring an apache server and some basic Linux commands. Once we have the application then we can figure out buying a domain name and hosting”
Myself — “Hmm! Sounds fun let me give it a try!”
Manish — “ Yeah…! it will be” (Smirking widely)
The conclusion from the story is, that full-stack development was TOUGH and time-consuming. It was not meant for everybody. In software development teams, there was a clear distinction between a front-end designer, backend engineer, Database Administrator, and the DevOps professional. All of them were masters of different programming languages and tools. Back then Full-stack-DevOps were treated like Gods as they were the real MVPs, Hackers, problem -solvers, the one-man army, the wizards.
That’s not the case ten years later today in 2022. The development ecosystem is heavily layered and complex, however, utilizing these frontend JS frameworks (React, Angular, etc), Nodejs, MongoDB, Docker containers and convenient cloud services for deployment and Hosting even a single person can create a complex web app. These days to get the God title, Full-Stack-DevOps should also be Cloud & System-Design experts, they should be able to design, develop, test, and deploy features in an enterprise-grade application themselves.
To make this happen, the following events were critical and led to the explosion of the JS ecosystem.
2008 — Google launched chrome and the most powerful JavaScript Engine, ‘V8’ world has ever seen. It was hundreds of times faster than the next best thing.
2009 — Two big things happened.
First, Ryan Dahl took the Open-sourced V8 codebase and added some more code on the top, and created Nodejs (The first-ever server-side run-time environment for JavaScript).
Second, the first version of the MongoDB database was shipped. Built on a JSON-like document data model with its interactive shell using a JavaScript interface.
2010 — Express and socket.io are born, and Angular is becoming popular.
2013 — JSON became the most popular data exchange format and beat XML in popularity.
2015 — By then we had matured versions of React, Angular, NPM, and Webpack for tooling.
2016 — Babeljs exploded into popularity and writing new JS but supporting old browsers was convenient. VSCode written in TypeScript became popular and to this day, it is the first preference of developers.
To sum it up, “JavaScript for front-end, JavaScript for backend, JavaScript for Database, JavaScript for tooling, JSON for configurations and development on a JS ecosystem-based IDE.”
It was so convenient and for the first time, I was able to teach someone Full-stack development within a few months. It meant that companies can quickly onboard and train Junior devs, Rapid prototyping is really fast and everyone is releasing MVPs in just weeks, beating the competition.
When using MongoDB, applications don’t have to worry about the Schema, Relationships, and data types upfront. We also don’t need to learn another programming language like SQL just to work with Data. MongoDB complemented the JavaScript ecosystem very well and in turn, became popular.
3. Acquisition of WiredTiger, an open-source, high-performance storage engine company by MongoDB.
The storage engine is the component of the database that is responsible for managing how data is stored, both in memory and on disk. MongoDB supports multiple storage engines, as different engines perform better for specific workloads.
In 2014, MongoDB acquired WiredTiger, a high-performance storage engine company co-founded and led by world-renowned experts in transactional processing Keith Bostic and Dr. Michael Cahill. Two of the major reasons for ignoring MongoDB by enterprise clients was the lack of transaction and encryption at rest support. Integrating WiredTiger with MongoDb was a game-changer for almost all workloads and opened up opportunities with big clients.
After integrating WiredTiger with MongoDb in version 3.0, they observed 7–10 times performance improvement and 80% less storage than previous engines.
- Keith Bostic, Co-founder of WiredTiger, Senior staff engineer at MongoDb
Ending Notes
I am a Full-stack developer and worked with MongoDb for an enterprise-grade product for around 3 years. I still enjoy writing MongoDb queries and creating MongoDB atlas instances for my projects. I love that it supports horizontal scaling right out of the box.
I never saw any article or posts talking about the historic milestones on MongoDB. Would love to get your feedback and highly recommend watching two videos — Launch of chrome and technical introduction to WiredTiger by Keith Bostic.
Please feel free to reach out anytime. https://linktr.ee/neetishraj
References
- MongoDB History: https://www.mongodb.com/company
- Mongo DB evolution: https://www.mongodb.com/evolved
- Elliot Horowitz on 10 years of MongoDB
- Google to acquire DoubleClick
- Google is bringing DoubleClick to billboard ads for the first time which could be huge for outdoor advertising
- Launch of chrome and V8 at Google event in 2008 (For V8, start at 44:20)
- JSON vs XML
- A brief history of Nodejs
- New storage engine, WiredTiger being introduced to MongoDB
- Mmap vs WiredTiger
- Technical introduction to WiredTiger, by Keith Bostic
- Path to transactions, discussion on integrating WiredTiger with MongoDb, by Dr. Michael Cahill
