Mar
14
2019
--

Microsoft open sources its data compression algorithm and hardware for the cloud

The amount of data that the big cloud computing providers now store is staggering, so it’s no surprise that most store all of this information as compressed data in some form or another — just like you used to zip your files back in the days of floppy disks, CD-ROMs and low-bandwidth connections. Typically, those systems are closely guarded secrets, but today, Microsoft open sourced the algorithm, hardware specification and Verilog source code for how it compresses data in its Azure cloud. The company is contributing all of this to the Open Compute Project (OCP).

Project Zipline, as Microsoft calls this project, can achieve 2x higher compression ratios compared to the standard Zlib-L4 64KB model. To do this, the algorithm — and its hardware implementation — were specifically tuned for the kind of large data sets Microsoft sees in its cloud. Because the system works at the systems level, there is virtually no overhead and Microsoft says that it is actually able to manage higher throughput rates and lower latency than other algorithms are currently able to achieve.

Microsoft stresses that it is also contributing the Verilog source code for register transfer language (RTL) — that is, the low-level code that makes this all work. “Contributing RTL at this level of detail as open source to OCP is industry leading,” Kushagra Vaid, the general manager for Azure hardware infrastructure, writes. “It sets a new precedent for driving frictionless collaboration in the OCP ecosystem for new technologies and opening the doors for hardware innovation at the silicon level.”

Microsoft is currently using this system in its own Azure cloud, but it is now also partnering with others in the Open Compute Project. Among these partners are Intel, AMD, Ampere, Arm, Marvell, SiFive, Broadcom, Fungible, Mellanox, NGD Systems, Pure Storage, Synopsys and Cadence.

“Over time, we anticipate Project Zipline compression technology will make its way into several market segments and usage models such as network data processing, smart SSDs, archival systems, cloud appliances, general purpose microprocessor, IoT, and edge devices,” writes Vaid.

Oct
31
2016
--

Microsoft open sources its next-gen cloud hardware design

80527689-6b4f-4ac8-bb93-25556376560c Microsoft today open sourced its next-gen hyperscale cloud hardware design and contributed it to the Open Compute Project (OCP). Microsoft joined the OCP, which also includes Facebook, Google, Intel, IBM, Rackspace and many other cloud vendors, back in 2014. Over the last two years, it already contributed a number of server, networking and data center designs. With this new contribution,… Read More

Oct
07
2015
--

Open Network Linux Simplifies Open Compute Project Switch Configuration

Datacenter in blue tint. Big Switch Networks, Facebook and NTT, announced today that they have come together to create a unified operating system called Open Network Linux for Open Compute Project‘s (OCP) switch hardware. While the name doesn’t exactly roll off the tongue, the project is designed to help companies, whether web scale-type companies like Facebook or others looking to take advantage of the… Read More

Jun
18
2014
--

Facebook’s New Open-Source Data Switch Technology Is Designed For Flexibility And Greater Control

Facebook's new open source top of rack switch broken down by component parts. Today Facebook announced new modularized top-of-rack networking switch technology and a new Linux OS to control these switches. The company will eventually share this technology as part of its Open Compute Project.
Jay Parikh, Facebook’s VP of Infrastructure Engineering, made the announcement at GigaOm’s Structure Conference in an interview with Derrick Harris. Read More

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com