Jun
10
2019
--

Qubole launches Quantum, its serverless database engine

Qubole, the data platform founded by Apache Hive creator and former head of Facebook’s Data Infrastructure team Ashish Thusoo, today announced the launch of Quantum, its first serverless offering.

Qubole may not necessarily be a household name, but its customers include the likes of Autodesk, Comcast, Lyft, Nextdoor and Zillow . For these users, Qubole has long offered a self-service platform that allowed their data scientists and engineers to build their AI, machine learning and analytics workflows on the public cloud of their choice. The platform sits on top of open-source technologies like Apache Spark, Presto and Kafka, for example.

Typically, enterprises have to provision a considerable amount of resources to give these platforms the resources they need. These resources often go unused and the infrastructure can quickly become complex.

Qubole already abstracts most of this away, offering what is essentially a serverless platform. With Quantum, however, it is going a step further by launching a high-performance serverless SQL engine that allows users to query petabytes of data with nothing else but ANSI-SQL, giving them the choice between using a Presto cluster or a serverless SQL engine to run their queries, for example.

The data can be stored on AWS and users won’t have to set up a second data lake or move their data to another platform to use the SQL engine. Quantum automatically scales up or down as needed, of course, and users can still work with the same metastore for their data, no matter whether they choose the clustered or serverless option. Indeed, Quantum is essentially just another SQL engine without Qubole’s overall suite of engines.

Typically, Qubole charges enterprises by compute minutes. When using Quantum, the company uses the same metric, but enterprises pay for the execution time of the query. “So instead of the Qubole compute units being associated with the number of minutes the cluster was up and running, it is associated with the Qubole compute units consumed by that particular query or that particular workload, which is even more fine-grained,” Thusoo explained. “This works really well when you have to do interactive workloads.”

Thusoo notes that Quantum is targeted at analysts who often need to perform interactive queries on data stored in object stores. Qubole integrates with services like Tableau and Looker (which Google is now in the process of acquiring). “They suddenly get access to very elastic compute capacity, but they are able to come through a very familiar user interface,” Thusoo noted.

 

Jun
05
2019
--

Google Cloud gets capacity reservations, extends committed use discounts beyond CPUs

Google Cloud made two significant pricing announcements today. Those, you’ll surely be sad to hear, don’t involve the usual price drops for compute and storage. Instead, Google Cloud today announced that it is extending its committed-use discounts, which give you a significant discount when you commit to using a certain number of resources for one or three years, to GPUs, Cloud TPU Pods and local SSDs. In return for locking yourself into a long-term plan, you can get discounts of 55% off on-demand prices.

In addition, Google is launching a capacity reservation system for Compute Engine that allows users to reserve resources in a specific zone for later use to ensure that they have guaranteed access to these resources when needed.

At first glance, capacity reservations may seem like a weird concept in the cloud. The promise of cloud computing, after all, is that you can just spin machines up and down at will — and never really have to think about availability.

So why launch a reservation system? “This is ideal for use cases like disaster recovery or peace of mind, so a customer knows that they have some extra resources, but also for retail events like Black Friday or Cyber Monday,” Google senior product manager Manish Dalwadi told me.

These users want to have absolute certainty that when they need the resources, they will be available to them. And while many of us think of the large clouds as having a virtually infinite amount of virtual machines available at any time, some machine types may occasionally only be available in a different availability zone, for example, that is not the same zone as where the rest of your compute resources are.

Users can create or delete reservations at any time and any existing discounts — including sustained use discounts and committed use discounts — will be applied automatically.

As for committed-use discounts, it’s worth noting that Google always took a pretty flexible approach to this. Users don’t have to commit to using a specific machine type for three years, for example. Instead, they commit to using a specific number of CPU cores and memory, for example.

“What we heard from customers was that other commit models are just too inflexible and their utilization rates were very low, like 70, 60% utilization,” Google product director Paul Nash told me. “So one of our design goals with committed-use discounts was to figure out how we could provide something that gives us the capacity planning signal that we need, provides the same amount of discounts that we want to pass on to customers, but do it in a way that customers actually feel like they are getting a great deal and so that they don’t have to hyper-manage these things in order to get the most out of them.”

Both the extended committed-use discounts and the new capacity reservation system for Compute Engine resources are now live in the Google Cloud.

Jun
05
2019
--

Microsoft and Oracle link up their clouds

Microsoft and Oracle announced a new alliance today that will see the two companies directly connect their clouds over a direct network connection so that their users can then move workloads and data seamlessly between the two. This alliance goes a bit beyond just basic direct connectivity and also includes identity interoperability.

This kind of alliance is relatively unusual between what are essentially competing clouds, but while Oracle wants to be seen as a major player in this space, it also realizes that it isn’t likely to get to the size of an AWS, Azure or Google Cloud anytime soon. For Oracle, this alliance means that its users can run services like the Oracle E-Business Suite and Oracle JD Edwards on Azure while still using an Oracle database in the Oracle cloud, for example. With that, Microsoft still gets to run the workloads and Oracle gets to do what it does best (though Azure users will also continue be able to run their Oracle databases in the Azure cloud, too).

“The Oracle Cloud offers a complete suite of integrated applications for sales, service, marketing, human resources, finance, supply chain and manufacturing, plus highly automated and secure Generation 2 infrastructure featuring the Oracle Autonomous Database,” said Don Johnson, executive vice president, Oracle Cloud Infrastructure (OCI), in today’s announcement. “Oracle and Microsoft have served enterprise customer needs for decades. With this alliance, our joint customers can migrate their entire set of existing applications to the cloud without having to re-architect anything, preserving the large investments they have already made.”

For now, the direct interconnect between the two clouds is limited to Azure US East and Oracle’s Ashburn data center. The two companies plan to expand this alliance to other regions in the future, though they remain mum on the details. It’ll support applications like JD Edwards EnterpriseOne, E-Business Suite, PeopleSoft, Oracle Retail and Hyperion on Azure, in combination with Oracle databases like RAC, Exadata and the Oracle Autonomous Database running in the Oracle Cloud.

“As the cloud of choice for the enterprise, with over 95% of the Fortune 500 using Azure, we have always been first and foremost focused on helping our customers thrive on their digital transformation journeys,” said Scott Guthrie, executive vice president of Microsoft’s Cloud and AI division. “With Oracle’s enterprise expertise, this alliance is a natural choice for us as we help our joint customers accelerate the migration of enterprise applications and databases to the public cloud.”

Today’s announcement also fits within a wider trend at Microsoft, which has recently started building a number of alliances with other large enterprise players, including its open data alliance with SAP and Adobe, as well as a somewhat unorthodox gaming partnership with Sony.

 

Jun
04
2019
--

How Kubernetes came to rule the world

Open source has become the de facto standard for building the software that underpins the complex infrastructure that runs everything from your favorite mobile apps to your company’s barely usable expense tool. Over the course of the last few years, a lot of new software is being deployed on top of Kubernetes, the tool for managing large server clusters running containers that Google open sourced five years ago.

Today, Kubernetes is the fastest growing open-source project and earlier this month, the bi-annual KubeCon+CloudNativeCon conference attracted almost 8,000 developers to sunny Barcelona, Spain, making the event the largest open-source conference in Europe yet.

To talk about how Kubernetes came to be, I sat down with Craig McLuckie, one of the co-founders of Kubernetes at Google (who then went on to his own startup, Heptio, which he sold to VMware); Tim Hockin, another Googler who was an early member on the project and was also on Google’s Borg team; and Gabe Monroy, who co-founded Deis, one of the first successful Kubernetes startups, and then sold it to Microsoft, where he is now the lead PM for Azure Container Compute (and often the public face of Microsoft’s efforts in this area).

Google’s cloud and the rise of containers

To set the stage a bit, it’s worth remembering where Google Cloud and container management were five years ago.

May
21
2019
--

Microsoft makes a push for service mesh interoperability

Services meshes. They are the hot new thing in the cloud native computing world. At KubeCon, the bi-annual festival of all things cloud native, Microsoft today announced that it is teaming up with a number of companies in this space to create a generic service mesh interface. This will make it easier for developers to adopt the concept without locking them into a specific technology.

In a world where the number of network endpoints continues to increase as developers launch new micro-services, containers and other systems at a rapid clip, they are making the network smarter again by handling encryption, traffic management and other functions so that the actual applications don’t have to worry about that. With a number of competing service mesh technologies, though, including the likes of Istio and Linkerd, developers currently have to choose which one of these to support.

“I’m really thrilled to see that we were able to pull together a pretty broad consortium of folks from across the industry to help us drive some interoperability in the service mesh space,” Gabe Monroy, Microsoft’s lead product manager for containers and the former CTO of Deis, told me. “This is obviously hot technology — and for good reasons. The cloud-native ecosystem is driving the need for smarter networks and smarter pipes and service mesh technology provides answers.”

The partners here include Buoyant, HashiCorp, Solo.io, Red Hat, AspenMesh, Weaveworks, Docker, Rancher, Pivotal, Kinvolk and VMware . That’s a pretty broad coalition, though it notably doesn’t include cloud heavyweights like Google, the company behind Istio, and AWS.

“In a rapidly evolving ecosystem, having a set of common standards is critical to preserving the best possible end-user experience,” said Idit Levine, founder and CEO of Solo.io. “This was the vision behind SuperGloo — to create an abstraction layer for consistency across different meshes, which led us to the release of Service Mesh Hub last week. We are excited to see service mesh adoption evolve into an industry-level initiative with the SMI specification.”

For the time being, the interoperability features focus on traffic policy, telemetry and traffic management. Monroy argues that these are the most pressing problems right now. He also stressed that this common interface still allows the different service mesh tools to innovate and that developers can always work directly with their APIs when needed. He also stressed that the Service Mesh Interface (SMI), as this new specification is called, does not provide any of its own implementations of these features. It only defines a common set of APIs.

Currently, the most well-known service mesh is probably Istio, which Google, IBM and Lyft launched about two years ago. SMI may just bring a bit more competition to this market since it will allow developers to bet on the overall idea of a service mesh instead of a specific implementation.

In addition to SMI, Microsoft also today announced a couple of other updates around its cloud-native and Kubernetes services. It announced the first alpha of the Helm 3 package manager, for example, as well as the 1.0 release of its Kubernetes extension for Visual Studio Code and the general availability of its AKS virtual nodes, using the open source Virtual Kubelet project.

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com