MongoDB on the four technical characteristics of IoT success

Jon Reed Profile picture for user jreed September 17, 2015
MongoDB reached out to me with good news about the Internet of Things. I challenged them to share real world use cases - they accepted. Here's what CEO Dev Ittycheria and VP Strategy and Product Marketing Kelly Stirman had to say.

double exposure of hand showing Internet of things (IoT) word di
It's been busy times for MongoDB since I last covered them at MongoDB World. One point of emphasis? MongoDB's role in the Internet of Things. After my own contribution to Internet of Things FUD, I was eager to get my skepticism challenged.

On a recent chat, CEO Dev Ittycheria and VP Strategy and Product Marketing Kelly Stirman did just that. We started with a MongoDB update. Since MongoDB world in June, MongoDB's been rounding out the management team. Michael Gordon signed on as CFO. Next up: a head of HR, or "head of people" as Ittycheria calls it.

These folks will join other (relatively) recent hires at CMO and Chief Revenue Officer (head of sales). Ittycheria's hiring imperative? Form a team that can scale and manage a business "a lot bigger than MongoDB is today." MongoDB's Q2 closed in July. The company doesn't disclose their financial results, but Ittycheria called the quarter "strong", and Q3 "looks to be a really strong quarter as well."

Meantime, on the product front, the team is building out 3.2 features highlighted in June (Ittycheria says release 3.2 is still targeted for end of 2015). "There's a lot of people with their heads down," he says.

Ittycheria acknowledged that constructing a team across North America, Europe and Asia is "not trivial." But he adds: "We feel like it's clear that we are the leader in the NoSQL space, and that there's a massive opportunity in front of us."

The four technical characteristics of IoT success

Which brings us to the Internet of Things. Stirman has a twist on IoT hype. He thinks some technology companies are hyping up the IoT beyond where they can execute. And - this part is not a surprise - he believes MongoDB is ideally suited to deliver on the IoT. I asked him to make his case. Stirman drills into four supporting points:

1. IoT means parsing mass amounts of data. Throwing hardware at the issue is financially problematic, because often the data surges are seasonal. Stirman:

IoT is about taking really large volumes and fast-arriving sources of data. This data may have little or no value on a record-by-record basis, but when you put the data together in aggregate, it starts to have a lot of value. To work with data like that at that's moving quickly and is at very large volume you need technology that's able to work with those kinds of demands.

Typically that means you need to be able to throw lots of hardware at the situation. The amount of hardware you are using can vary depending on seasonality, or time of day, or an important event. If you're a city trying to do some sort of a smart city initiative, things like a major sporting event, or a bad weather system, or a parade of some kind can really stress the infrastructure. Being able to scale elastically and accommodate very large volumes of fast moving data is one piece of the equation.

2. With the proliferation of new data sources, from sensors to smart devices, your IoT system must be able to adapt flexibly:

What you want is a system that can flexibly adapt to new sources of input and evolve gracefully to accommodate those sources - without taking the system offline, and without incurring lots of engineering overhead to make changes. That means you need a very flexible and dynamic data model.

3. Most IoT apps are deployed in a cloud environment. Companies need software that can manage global scenarios:

That means being able to scale horizontally, but also being able to deploy across multiple data centers globally, around a sort of a global database. Chances are your network of sensors isn't just in your local neighborhood or within a couple of miles of your data center. Chances are it's all over the planet, so you need a system that works in that way.

4. It's not about collecting the data. It's about deriving real world insight. That means real-time analysis:

In the beginning, each of these records on its own might not have an enormous amount of value. It's looking at it in aggregate where you have the power. You need a system that can not only capture the data but give you the ability to run analytics in real time as the data is arriving.

Needless to say, Stirman believes MongoDB excels in all four of these IoT areas. He believes that when customers try to put these four capabilities together, most solutions fall short. His examples:

  • Hadoop falls short on real-time analytics. Even with Spark? Stirman: "Spark is a big step forward from traditional MapReduce and it's very, very useful - especially for retrospective machine learning type applications, but it's still not going to get you sub-second latency on fast moving data. It's just not designed for that." Though Stirman admits that Spark does plenty of things MongoDB doesn't do, which is why he sees it as a complementary solution. "We're seeing a lot of people running Spark on top of Mongo," he says.
  • As for "traditional" relational databases, Stirman acknowledges that such databases can accommodate fast-moving data, "If you know the dark magic of making a relational database fast, which most people don't." But he doesn't think relational databases are equipped to handle the rapidly changing data models IoT requires. Yes, you can run relational databases in the cloud, but you can't deploy it across data centers. "You've painted yourself into a corner."

IoT in the field - MongoDB use cases

OK - so MongoDB thinks it's in the IoT sweet spot. But what do customers think? I asked Stirman and Ittycheria for some customer use cases besides Bosch, which is a well-known MongoDB IoT use case that has gotten a lot of play.

Ittycheria cited Silver Spring Networks, a provider of smart energy networks. Utilities are finding that about seventy percent of their data is generated from smart grid networks, but extracting the data from those networks is, as Ittycheria puts it, "incredibly painful." To solve this, Silver Spring built a sensor network app on MongoDB that captures high volume data. Ittycheria:

To Kelly's point, it's a lot of data coming from a lot of different sources. The app provides real-time analytics based on geospatial and sensor types. Silver Spring uses this data for understanding usage patterns, ensuring they're prepared for spikes in energy usage or any resource constraints that they have.

Ittycheria also brought up another energy customer, EnerNOC. EnerNOC is a capacity planning tool for electricity. They use sensor data to determine where the demand is company from and the supply available:

They are now able to provide transparency on where there's excess or reserve capacity versus where there is high demand. They essentially have, I believe, almost 1.5 billion data points that they capture per month to essentially capture and track usage and where capacity is available. Essentially they allow people to drive more efficiency in terms of when they use energy and how they use energy. It's a pretty sophisticated application, again, built on Mongo.

Ittycheria also cited another customer, an industrial equipment manufacturer that now has sensors on their equipment. They are now able to track patterns around the nature of the soil, impending weather, and other factors. Like many IoT plays, the data itself is becoming a business:

Once they've captured that data, they sell it back to consumers or users of their technology to better plan how they work their land.

Stirman sees this example as indicative of a broader need for IoT across industries, addressing fleet management:

We've seen this time and again with different companies that are in the business of managing fleets, whether it's ships, or boats, or planes, or what have you. Eventually, it'll be drones. It's the same type of scenario: I have lots of sensors, lots of inputs, and I want real-time visibility into that infrastructure so I can do a better job of managing for safety and efficiency.

He brought up one other example: the city of Chicago and its WindyGrid app. Chicago built WindyGrid on MongoDB which provides color-coded information about the quality of services and pending problems:

They're pulling in data from the traffic lights, from the garbage trucks, from the 911 calls, from citizens reporting issues, from the weather - dozens of data sources. The lowest common denominator is location and time. You put it all on a map and then you can see, "Well, there's this emerging issue around sewer repair on this particular block, and we also know that the garbage trucks are going to go down there later today, so we need to reroute things and change the traffic signals.

The city of Chicago is taking WindyCity one step further: they plan to open source the application so other municipalities can build and improve upon it.

Final thoughts

MongoDB is hardly the only company that sees itself in the center of the IoT play. One thing we did not have time for was a deeper discussion on in-memory databases. It strikes me that in-memory databases are ideal for some of these IoT needs, as long as they can scale affordably. From what I can tell, MongoDB seems to be thinking along the same lines. I expect we'll hear more from them on their own in-memory plans in the next twelve months.

It was good to hear about the use cases on the ground; I plan to do more direct interviews with IoT projects from MongoDB and other vendors in the months to follow. MongoDB clearly has a solid approach to ease of ramp up and app building. How far that developer engagement edge will drive their success amidst the deeper pockets of the database old guard remains to be seen.

This will be a fun story to watch. Like all these "disruptive" narratives, the real winner is database customers, who can look ahead to a wider range of options and, I'd argue, much more compelling use cases.


Image credit - double exposure of hand showing Internet of things (IoT) word © everythingpossible -

Disclosure - MongoDB has no financial ties to diginomica, though they did pay my travel expenses to attend MongoDB World in June. I was approached by MongoDB PR about this story, and thought it was worth a listen.

A grey colored placeholder image