Sep
24
2018
--

Microsoft updates its planet-scale Cosmos DB database service

Cosmos DB is undoubtedly one of the most interesting products in Microsoft’s Azure portfolio. It’s a fully managed, globally distributed multi-model database that offers throughput guarantees, a number of different consistency models and high read and write availability guarantees. Now that’s a mouthful, but basically, it means that developers can build a truly global product, write database updates to Cosmos DB and rest assured that every other user across the world will see those updates within 20 milliseconds or so. And to write their applications, they can pretend that Cosmos DB is a SQL- or MongoDB-compatible database, for example.

CosmosDB officially launched in May 2017, though in many ways it’s an evolution of Microsoft’s existing Document DB product, which was far less flexible. Today, a lot of Microsoft’s own products run on CosmosDB, including the Azure Portal itself, as well as Skype, Office 365 and Xbox.

Today, Microsoft is extending Cosmos DB with the launch of its multi-master replication feature into general availability, as well as support for the Cassandra API, giving developers yet another option to bring existing products to CosmosDB, which in this case are those written for Cassandra.

Microsoft now also promises 99.999 percent read and write availability. Previously, it’s read availability promise was 99.99 percent. And while that may not seem like a big difference, it does show that after more of a year of operating Cosmos DB with customers, Microsoft now feels more confident that it’s a highly stable system. In addition, Microsoft is also updating its write latency SLA and now promises less than 10 milliseconds at the 99th percentile.

“If you have write-heavy workloads, spanning multiple geos, and you need this near real-time ingest of your data, this becomes extremely attractive for IoT, web, mobile gaming scenarios,” Microsoft CosmosDB architect and product manager Rimma Nehme told me. She also stressed that she believes Microsoft’s SLA definitions are far more stringent than those of its competitors.

The highlight of the update, though, is multi-master replication. “We believe that we’re really the first operational database out there in the marketplace that runs on such a scale and will enable globally scalable multi-master available to the customers,” Nehme said. “The underlying protocols were designed to be multi-master from the very beginning.”

Why is this such a big deal? With this, developers can designate every region they run Cosmos DB in as a master in its own right, making for a far more scalable system in terms of being able to write updates to the database. There’s no need to first write to a single master node, which may be far away, and then have that node push the update to every other region. Instead, applications can write to the nearest region, and Cosmos DB handles everything from there. If there are conflicts, the user can decide how those should be resolved based on their own needs.

Nehme noted that all of this still plays well with CosmosDB’s existing set of consistency models. If you don’t spend your days thinking about database consistency models, then this may sound arcane, but there’s a whole area of computer science that focuses on little else but how to best handle a scenario where two users virtually simultaneously try to change the same cell in a distributed database.

Unlike other databases, Cosmos DB allows for a variety of consistency models, ranging from strong to eventual, with three intermediary models. And it actually turns out that most CosmosDB users opt for one of those intermediary models.

Interestingly, when I talked to Leslie Lamport, the Turing award winner who developed some of the fundamental concepts behind these consistency models (and the popular LaTeX document preparation system), he wasn’t all that sure that the developers are making the right choice. “I don’t know whether they really understand the consequences or whether their customers are going to be in for some surprises,” he told me. “If they’re smart, they are getting just the amount of consistency that they need. If they’re not smart, it means they’re trying to gain some efficiency and their users might not be happy about that.” He noted that when you give up strong consistency, it’s often hard to understand what exactly is happening.

But strong consistency comes with its drawbacks, too, which leads to higher latency. “For strong consistency there are a certain number of roundtrip message delays that you can’t avoid,” Lamport noted.

The CosmosDB team isn’t just building on some of the fundamental work Lamport did around databases, but it’s also making extensive use of TLA+, the formal specification language Lamport developed in the late 90s. Microsoft, as well as Amazon and others, are now training their engineers to use TLA+ to describe their algorithms mathematically before they implement them in whatever language they prefer.

“Because [CosmosDB is] a massively complicated system, there is no way to ensure the correctness of it because we are humans, and trying to hold all of these failure conditions and the complexity in any one person’s — one engineer’s — head, is impossible,” Microsoft Technical Follow Dharma Shukla noted. “TLA+ is huge in terms of getting the design done correctly, specified and validated using the TLA+ tools even before a single line of code is written. You cover all of those hundreds of thousands of edge cases that can potentially lead to data loss or availability loss, or race conditions that you had never thought about, but that two or three years ago after you have deployed the code can lead to some data corruption for customers. That would be disastrous.”

“Programming languages have a very precise goal, which is to be able to write code. And the thing that I’ve been saying over and over again is that programming is more than just coding,” Lamport added. “It’s not just coding, that’s the easy part of programming. The hard part of programming is getting the algorithms right.”

Lamport also noted that he deliberately chose to make TLA+ look like mathematics, not like another programming languages. “It really forces people to think above the code level,” Lamport noted and added that engineers often tell him that it changes the way they think.

As for those companies that don’t use TLA+ or a similar methodology, Lamport says he’s worried. “I’m really comforted that [Microsoft] is using TLA+ because I don’t see how anyone could do it without using that kind of mathematical thinking — and I worry about what the other systems that we wind up using built by other organizations — I worry about how reliable they are.”

more Microsoft Ignite 2018 coverage

Apr
23
2018
--

Tableau gets new pricing plans and a data preparation tool

Data analytics platform Tableau today announced the launch of both a new data preparation product and a new subscription pricing plan.

Currently, Tableau offers desktop plans for users who want to analyze their data locally, a server plan for businesses that want to deploy the service on-premises or on a cloud platform, and a fully hosted online plan. Prices for these range from $35 to $70 per user and month. The new pricing plans don’t focus so much on where the data is analyzed but on the analyst’s role. The new Creator, Explorer and Viewer plans are tailored toward the different user experiences. They all include access to the new Tableau Prep data preparation tool, Tableau Desktop and new web authoring capabilities — and they are available both on premises or in the cloud.

Existing users can switch their server or desktop subscriptions to the new release today and then assign each user either a creator, explorer or viewer role. As the name indicates, the new viewer role is meant for users who mostly consume dashboards and visualizations, but don’t create their own. The explorer role is for those who need access to a pre-defined data set and the creator role is for analysts and power user who need access to all of Tableau’s capabilities.

“Organizations are facing the urgent need to empower their entire workforce to help drive more revenue, reduce costs, provide better service, increase productivity, discover the next scientific breakthrough and even save lives,” said Adam Selipsky, CEO at Tableau, in today’s announcement. “Our new offerings will help entire organizations make analytics ubiquitous, enabling them to tailor the capabilities required for every employee.”

As for the new data preparation tool, the general idea here is to give users a visual way to shape and clean their data, something that’s especially important as businesses now often pull in data from a variety of sources. Tableau Prep can automate some of this, but the most important aspect of the service is that it gives users a visual interface for creating these kind of workflows. Prep includes support for all the standard Tableau data connectors and lets users perform calculations, too.

“Our customers often tell us that they love working with Tableau, but struggle when data is in the wrong shape for analysis,” said Francois Ajenstat, Chief Product Officer at Tableau. “We believe data prep and data analysis are two sides of the same coin that should be deeply integrated and look forward to bringing fun, easy data prep to everyone regardless of technical skill set.”

Mar
21
2018
--

Microsoft Power Apps update includes new Common Data Service

Microsoft announced the spring update to its Power BI and Power Apps platform today with a significant enhancement, a new common data service that enables companies to build data-based applications from a variety of data sources.

This is part of a wider strategy that is designed to remove some of the complexity associated with gathering, processing and incorporating data into applications.

Microsoft is essentially giving customers access to the same set of tools and services it has used internally to build Dynamics 365, its enterprise suite of tools that includes CRM, marketing automation and field service along with enterprise resource planning tools (ERP).

While the company has been allowing third party developers to build application on the platform for about 18 months with its Power Apps tools, they haven’t been able to take advantage of the data under the hood without some heavy lifting. Microsoft aims to change that with the Common Data Service.

Diagram: Microsoft

“What that service means, practically speaking, is that it’s not only a place to store data, but a model (schema) that is stamped out there with everything you would need to build a business app around [elements] such as contacts, events, customers [and so forth], Ryan Cunningham, Microsoft program manager for Power Apps explained. This allows the programmer to take advantage of pre-built relationships and rules and how they should be enforced without having to code them from scratch.

Cunningham points out that they tried to make it fairly simple to build the apps, while still providing a level of customization and the ability to use Microsoft data or data from another source. That’s where the Common Data Store comes in.

He says that developers can take advantage of the 200 connectors that come pre-built out of the box and connect to all that data you’ve been collecting in the Microsoft products, but they aren’t limited to the Microsoft data. “You can still build custom applications on top of the platform, and get the benefit of the platform we’ve built our tools on,” he said.

The Common Data Store is part of a much broader set of announcements around the spring releases of Dynamics 365, Office 365 and Power BI platforms all announced today.

Mar
06
2018
--

Data.world introduces enterprise data collaboration platform

 Imagine a tool that’s a kind of Facebook for data inside large organizations. You could build data projects and teams, upload and share data sets, then discuss your raw data and findings with colleagues in a community setting. That’s precisely what Data.world, an Austin startup, released today. Data is the lifeblood of most modern organizations and Data.world has tried to build a… Read More

Feb
21
2018
--

Crunchbase opens a marketplace for 3rd-party data in bid to be the ‘master database for companies’

 Last year, when tech-company database Crunchbase (once a part of TechCrunch but since spun out) announced a funding round of $18 million, it previewed plans for a new marketplace where it would sell access to third-party data to supplement its own information. Now nearly one year later, it’s taking the wraps off that project. Read More

Feb
13
2018
--

Pulse Q&A wants to transform the way we gather data from CIOs

 Lots of companies need to understand what CIOs are thinking, but it’s hard to get a group of busy people to give meaningful answers about the products they use or their budget priorities in public forums for obvious reasons. Pulse Q&A is a new company in the Y Combinator Winter 2018 class that wants to change how we gather and share this valuable information. “Imagine you had… Read More

Dec
06
2017
--

Inflect raises $3M seed round to make buying internet infrastructure easier

 Inflect, a startup that wants to make it easier for businesses to buy their own internet infrastructure, today announced that it has raised a $3 million seed funding round. The service, which is still in preview, provides business with the necessary data to make their purchasing decisions when they go out and look for their own data center space, networking services and exchange… Read More

Oct
24
2017
--

Primer helps governments and corporations monitor and understand the world’s information

 When Google was founded in 1998, its goal was to organize the world’s information. And for the most part, mission accomplished — but in 19 years the goalpost has moved forward and indexing and usefully presenting information isn’t enough. As machine learning matures, it’s becoming feasible for the first time to actually summarize and contextualize the world’s… Read More

Sep
26
2017
--

Microsoft Excel is about to get a lot smarter

 Microsoft Excel users rejoice — your favorite spreadsheet is about to get a lot smarter. Thanks to the help of machine learning and a better connection to the outside world, Excel will soon be able to understand more about your inputs and then pull additional information from the internet as necessary. Read More

Sep
14
2017
--

Looker’s latest looks to simplify data integrations

 Looker is holding its Join user conference this week, and these affairs often involve a new release to show off to customers. Today, the company released Looker 5, which they say will make it easier for employees to make use of data in their work lives. Company CEO Frank Bien believes there is a growing group of people, who need quick access to data to do their jobs. “At a high level,… Read More

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com