The Cloud Native Computing Foundation adds etcd to its open-source stable

The Cloud Native Computing Foundation (CNCF), the open-source home of projects like Kubernetes and Vitess, today announced that its technical committee has voted to bring a new project on board. That project is etcd, the distributed key-value store that was first developed by CoreOS (now owned by Red Hat, which in turn will soon be owned by IBM). Red Hat has now contributed this project to the CNCF.

Etcd, which is written in Go, is already a major component of many Kubernetes deployments, where it functions as a source of truth for coordinating clusters and managing the state of the system. Other open-source projects that use etcd include Cloud Foundry, and companies that use it in production include Alibaba, ING, Pinterest, Uber, The New York Times and Nordstrom.

“Kubernetes and many other projects like Cloud Foundry depend on etcd for reliable data storage. We’re excited to have etcd join CNCF as an incubation project and look forward to cultivating its community by improving its technical documentation, governance and more,” said Chris Aniszczyk, COO of CNCF, in today’s announcement. “Etcd is a fantastic addition to our community of projects.”

Today, etcd has well over 450 contributors and nine maintainers from eight different companies. The fact that it ended up at the CNCF is only logical, given that the foundation is also the host of Kubernetes. With this, the CNCF now plays host to 17 projects that fall under its “incubated technologies” umbrella. In addition to etcd, these include OpenTracing, Fluentd, Linkerd, gRPC, CoreDNS, containerd, rkt, CNI, Jaeger, Notary, TUF, Vitess, NATS Helm, Rook and Harbor. Kubernetes, Prometheus and Envoy have already graduated from this incubation stage.

That’s a lot of projects for one foundation to manage, but the CNCF community is also extraordinarily large. This week alone about 8,000 developers are converging on Seattle for KubeCon/CloudNativeCon, the organization’s biggest event yet, to talk all things containers. It surely helps that the CNCF has managed to bring competitors like AWS, Microsoft, Google, IBM and Oracle under a single roof to collaboratively work on building these new technologies. There is a risk of losing focus here, though, something that happened to the OpenStack project when it went through a similar growth and hype phase. It’ll be interesting to see how the CNCF will manage this as it brings on more projects (with Istio, the increasingly popular service mesh, being a likely candidate for coming over to the CNCF as well).


Clarifai rolls out an on-premise visual search tool for businesses

Clarifai has traditionally been known as a web-based visual search tool that developers can integrate into their services. But as more and more businesses start to get onto the service, with their own specific demands — like getting them in-house — the New York-based startup has to grow up a bit.

Today, the company is starting a few early steps to do just that. After raising $30 million in October 2016, the company said today that it’s going to start rolling out an on-premise solution for larger enterprises where they can use all of Clarifai’s services in-house without that data shipping outside of their own servers. Larger businesses often have their own kinds of proprietary data and customers to work with, which have myriad different demands, and sometimes have to make sure their customer data doesn’t leave their close watch — or it needs to happen so fast that it can’t afford the lag of connecting to the cloud.

“Think of a casino with 10,000 cameras that all go to a central place,” CEO Matt Zeiler said. “You have like a streaming service that’s [faster than] running through the cloud that needs to run close to the edge. The camera is there, it’s not good enough, so you have to stream it to a central hub. You have to close the latency down, scale it, and protect privacy. It’s all the same API objects and structures you would use.”

The company also said it made two hires at high-level executive spots: a VP of engineering and a VP of product. Ulas Bardak joined Clarifai to head engineering after working at Whisper, while Rajesh Talpade previously served as a product lead for Google . Both of these hires are part of that growing-up process as it starts to try to woo more businesses — and the kinds of talent it needs to build these tools.

Part of Clarifai’s pitch is that it can be a neutral party when it comes to image recognition. While companies like Google and Pinterest can have their own sets of data and build custom visual search algorithms for their products, Clarifai works to build those tools that other companies can tap in order to have their own kinds of visual search tools. It’s part of a process to hand off some of these harder processes to a third party and focus on the key elements of their products, a hallmark of web development that led to an explosion in the startup ecosystem.

“When you use something like us, our research team is continually improving algorithms, and continuing to collect data, and that’ll give you a better level of accuracy and improve over time,” Zeiler said. “That’s a huge advantage and it gets away from how fast you train. We want it to be continuously improving. That takes a lot of resources and data, but that’s the big advantage for signing up for a service like us. We’re always comparing them in terms of quantitative elements like accuracy and qualitative elements like the team that reviews the models before they get published to make sure they’re not making stupid mistakes.”

To cap all this off, Clarifai also is making a bit of a rebrand of the website. That’s a small touch, but it all adds up to ways to help articulate to businesses exactly what they do and that they are building the tools necessary to become a go-to tool for developers when it comes to visual search.

Clarifai, of course, stands at constant risk from some of these larger players and the chance that they’ll just open up their visual search algorithms for developers. There are obvious candidates, but at the same time, if they are trying to build a walled garden they probably want to keep that data and those tools to themselves. If that remains the case, it may buy Clarifai enough time to collect enough data, have all the elements of any given image or video marked up and offer a compelling enough case for businesses to adopt it.


Clarifai rolls out an on-premise visual search tool for businesses

 Clarifai has traditionally been known as a web-based visual search tool that developers can integrate into their services. But as more and more businesses start to get onto the service, with their own specific demands — like getting them in-house — the New York-based startup has to grow up a bit.
Today, the company is starting a few early steps to do just that. After raising $30… Read More


Percona Live 2016: Operational Buddhism — Building Reliable Services From Unreliable Components

percona live 2016

It’s another packed day here at Percona Live 2016, with many, many database topics under discussion. Some technical, some strategical, and some operational. One such talk I sat in on was given by Ernie Souhrada, Database Engineer and Bit Wrangler at Pinterest. His talk was called Operational Buddhism: Building Reliable Services From Unreliable Components.

In it he discussed how the rise of utility computing has revolutionized much about the way organizations think about infrastructure and back-end serving systems, compared to the “olden days” of physical data centers. But success is still driven by meeting your SLAs. If services are up and sufficiently performant, you win. If not, you lose. In the traditional data center environment, fighting the uptime battle was typically driven by a philosophy Ernie calls “Operational Materialism.” The primary goal of OM is preventing failures at the infrastructure layer, and mechanisms for making this happen are plentiful and well-understood, many of which boil down to simply spending enough money to have at least N+1 of anything that might fail.

Ernie contends that in the cloud, Operational Materialism cannot succeed. Although the typical cloud provider tends to be holistically reliable, there are no guarantees that any individual virtual instance will not randomly or intermittently drop off the network or be terminated outright. Since we still need to keep our services up and running and meet our SLAs, we need a different mindset that accounts for the fundamentally opaque and ephemeral nature of the public cloud.

Ernie presented an alternative to OM, a worldview that he referred to as “Operational Buddhism.” Like traditional Buddhism, OB has Four Noble Truths:

  1. Cloud-based servers can fail at any time for any reason
  2. Trying to prevent this server failure is an endless source of suffering for DBAs and SREs alike
  3. Accepting the impermanence of individual servers, we can focus on designing systems that are failure-resilient, rather than failure-resistant
  4. We can escape the cycle of suffering and create a better experience for our customers, users, and colleagues.

To illustrate these concepts with concrete examples, he discussed how configuration management, automation, and service discovery help Pinterest to practice Operational Buddhism for both stateful (MySQL, HBase) and stateless (web) services. He also talked about some of the roads not taken, including the debate over Infrastructure-as-a-Service (IaaS) vs. Platform-as-a-Service (PaaS).

I was able to have a quick chat with Ernie after the talk:


See the rest of the Percona Live 2016 schedule here.


Percona Live MySQL Conference and Expo 2013: It feels like 2007 again

MySQL Banner: Keynote stageI actually don’t remember exactly whether it was in 2006, 2007 or 2008 — but around that time the MySQL community had one of the greatest MySQL conferences put on by O’Reilly and MySQL. It was a good, stable, predictable time.

Shortly thereafter, the MySQL world saw acquisitions, forks, times of uncertainly, more acquisitions, more forks, rumors (“Oracle is going to kill MySQL and the whole Internet”) and just a lot of drama and politics.

And now, after all this time some 6 or 7 years later, it feels like a MySQL Renaissance. All of the major MySQL players are coming to the Percona Live MySQL Conference and Expo 2013. I am happy to see Oracle’s engineers coming with talks — and now with a great MySQL 5.6 release — and I have great confidence in MySQL’s future with Oracle’s commitment level, coupled with the fact that Percona Server and MariaDB are continuing to mature.

For this year’s conference I am going to give a tutorial titled, “Percona XtraBackup: Old and New Features.” I actually traded my other talk, “MySQL 5.6 improvements from InnoDB internals prospective,” with Oracle’s Sunny Bains and his talk, MySQL 5.6: What’s New in InnoDB. Sunny honestly knows more about InnoDB internals than I do.

There are going to be a lot of great talks, and I would have a hard time listing all of the ones on my list to attend — but you can check all of them out for yourself. But I will say that the greatest part of the conference is not talks, but what happens in between them — in the conference center halls and behind the curtains.

If you are lucky you can grab engineers from Facebook, Twitter, Google, Box, Esty, Pinterest, Tumblr,… and ask them any question and actually get an answer. You can even catch interesting rumors… and spread some of your own. Basically just be part of the MySQL crowd and really feel the great vibe (especially after a few drinks during the Community Networking Reception).

I am personally looking forward to another lengthy discussion with Dimitri Kravtchuk on how benchmarks should be run, what the next bottleneck in MySQL will be, and how we, with Percona Server, are going to beat MySQL in performance. And you can actually go and ask Sunny anything about InnoDB internals: He is one of the few engineers in the world who knows everything about InnoDB.

If you seriously work with MySQL, then you should not miss the Percona Live MySQL Conference and Expo 2013. It runs April 22-25 in Santa Clara, California. But just remember: Advanced Rate tickets end on March 24th, so make sure you register this week.

The post Percona Live MySQL Conference and Expo 2013: It feels like 2007 again appeared first on MySQL Performance Blog.

Powered by WordPress | Theme: Aeros 2.0 by