Personal blog of Yzmir Ramirez

Docker and AWS today announced a new collaboration that introduces a deep integration between Docker’s Compose and Desktop developer tools and AWS’s Elastic Container Service (ECS) and ECS on AWS Fargate. Previously, the two companies note, the workflow to take Compose files and run them on ECS was often challenging for developers. Now, the two companies simplified this process to make switching between running containers locally and on ECS far easier.

“With a large number of containers being built using Docker, we’re very excited to work with Docker to simplify the developer’s experience of building and deploying containerized applications to AWS,” said Deepak Singh, the VP for compute services at AWS. “Now customers can easily deploy their containerized applications from their local Docker environment straight to Amazon ECS. This accelerated path to modern application development and deployment allows customers to focus more effort on the unique value of their applications, and less time on figuring out how to deploy to the cloud.”

In a bit of a surprise move, Docker last year sold off its enterprise business to Mirantis to solely focus on cloud-native developer experiences.

“In November, we separated the enterprise business, which was very much focused on operations, CXOs and a direct sales model, and we sold that business to Mirantis,” Docker CEO Scott Johnston told TechCrunch’s Ron Miller earlier this year. “At that point, we decided to focus the remaining business back on developers, which was really Docker’s purpose back in 2013 and 2014.”

Today’s move is an example of this new focus, given that the workflow issues this partnership addresses had been around for quite a while already.

It’s worth noting that Docker also recently engaged in a strategic partnership with Microsoft to integrate the Docker developer experience with Azure’s Container Instances.

AWS today announced the beta launch of Amazon Honeycode, a new, fully managed low-code/no-code development tool that aims to make it easy for anybody in a company to build their own applications. All of this, of course, is backed by a database in AWS and a web-based, drag-and-drop interface builder.

Developers can build applications for up to 20 users for free. After that, they pay per user and for the storage their applications take up.

Image Credits: Amazon/AWS

“Customers have told us that the need for custom applications far outstrips the capacity of developers to create them,” said AWS VP Larry Augustin in the announcement. “Now with Amazon Honeycode, almost anyone can create powerful custom mobile and web applications without the need to write code.”

Like similar tools, Honeycode provides users with a set of templates for common use cases like to-do list applications, customer trackers, surveys, schedules and inventory management. Traditionally, AWS argues, a lot of businesses have relied on shared spreadsheets to do these things.

“Customers try to solve for the static nature of spreadsheets by emailing them back and forth, but all of the emailing just compounds the inefficiency because email is slow, doesn’t scale, and introduces versioning and data syncing errors,” the company notes in today’s announcement. “As a result, people often prefer having custom applications built, but the demand for custom programming often outstrips developer capacity, creating a situation where teams either need to wait for developers to free up or have to hire expensive consultants to build applications.”

It’s no surprise then that Honeycode uses a spreadsheet view as its core data interface, which makes sense, given how familiar virtually every potential user is with this concept. To manipulate data, users can work with standard spreadsheet-style formulas, which seems to be about the closest the service gets to actual programming. ‘Builders,” as AWS calls Honeycode users, can also set up notifications, reminders and approval workflows within the service.

AWS says these databases can easily scale up to 100,000 rows per workbook. With this, AWS argues, users can then focus on building their applications without having to worry about the underlying infrastructure.

As of now, it doesn’t look like users will be able to bring in any outside data sources, though that may still be on the company’s roadmap. On the other hand, these kinds of integrations would also complicate the process of building an app and it looks like AWS is trying to keep things simple for now.

Honeycode currently only runs in the AWS US West region in Oregon but is coming to other regions soon.

Among Honeycode’s first customers are SmugMug and Slack.

“We’re excited about the opportunity that Amazon Honeycode creates for teams to build apps to drive and adapt to today’s ever-changing business landscape,” said Brad Armstrong, VP of Business and Corporate Development at Slack in today’s release. “We see Amazon Honeycode as a great complement and extension to Slack and are excited about the opportunity to work together to create ways for our joint customers to work more efficiently and to do more with their data than ever before.”

Slack and Amazon announced a big integration late yesterday afternoon. As part of the deal, Slack will use Amazon Chime for its call feature, while reiterating its commitment to use AWS as its preferred cloud provider to run its infrastructure. At the same time, Amazon has agreed to offer Slack as an option for all internal communications.

“Some parts of Amazon had licensed Slack before, but this is the first time it will be offered as an option to all employees,” an Amazon spokesperson told TechCrunch.

Make no mistake, this is a big deal as the SaaS communications tool increases its ties with AWS, but this agreement could also be about slighting Microsoft and its rival Teams product by making a deal with a cloud rival. In the past, Slack CEO Stewart Butterfield has had choice words for Microsoft saying the Redmond technology giant sees his company as an “existential threat.”

Whether that’s true  — Teams is but one piece of a huge technology company — it’s impossible not to look at the deal in this context. Aligning more deeply with AWS sends a message to Microsoft, whose Azure infrastructure services compete with AWS.

Butterfield didn’t say that of course. He talked about how synergistic the deal was. “Strategically partnering with AWS allows both companies to scale to meet demand and deliver enterprise-grade offerings to our customers. By integrating AWS services with Slack’s channel-based messaging platform, we’re helping teams easily and seamlessly manage their cloud infrastructure projects and launch cloud-based services without ever leaving Slack,” he said in a statement.

The deal also includes several other elements including integrating AWS Key Management Service with Slack Enterprise Key Management (EKM) for encryption key management, deeper alignment with AWS’s chatbot service and direct integration with AWS AppFlow to enable secure transfer of data between Slack and Amazon S3 storage and the Amazon Redshift data warehouse.

AWS CEO Andy Jassy saw it as a pure integration play. “Together, AWS and Slack are giving developer teams the ability to collaborate and innovate faster on the front end with applications, while giving them the ability to efficiently manage their backend cloud infrastructure,” Jassy said in a statement.

Like any good deal, it’s good for both sides. Slack gets a big customer in AWS and AWS now has Slack directly integrating more of its services. One of the reasons enterprise users are so enamored with Slack is the ability to get work done in a single place without constantly having to change focus and move between interfaces.

This deal will provide more of that for common customers, while tweaking a common rival. That’s what you call win-win.

It’s fair to say that even before the impact of COVID-19, companies had begun a steady march to the cloud. Maybe it wasn’t fast enough for AWS, as Andy Jassy made clear in his 2019 Re:invent keynote, but it was happening all the same and the steady revenue increases across the cloud infrastructure market bore that out.

As we look at the most recent quarter’s earnings reports for the main players in the market, it seems the pandemic and economic fall out has done little to slow that down. In fact, it may be contributing to its growth.

According to numbers supplied by Synergy Research, the cloud infrastructure market totaled $29 billion in revenue for Q12020.

Image Credit: Synergy Research

Synergy’s John Dinsdale, who has been watching this market for a long time, says that the pandemic could be contributing to some of that growth, at least modestly. In spite of the numbers, he doesn’t necessarily see these companies getting out of this unscathed either, but as companies shift operations from offices, it could be part of the reason for the increased demand we saw in the first quarter.

“For sure, the pandemic is causing some issues for cloud providers, but in uncertain times, the public cloud is providing flexibility and a safe haven for enterprises that are struggling to maintain normal operations. Cloud provider revenues continue to grow at truly impressive rates, with AWS and Azure in aggregate now having an annual revenue run rate of well over $60 billion,” Dinsdale said in a statement.

AWS led the way with a third of the market or more than $10 billion in quarterly revenue as it continues to hold a substantial lead in market share. Microsoft was in second, growing at a brisker 59% for 18% of the market. While Microsoft doesn’t break out its numbers, using Synergy’s numbers, that would work out to around $5.2 billion for Azure revenue. Meanwhile Google came in third with $2.78 billion.

If you’re keeping track of market share at home, it comes out to 32% for AWS, 18% for Microsoft and 8% for Google. This split has remained fairly steady, although Microsoft has managed to gain a few percentage points over the last several quarters as its overall growth rate outpaces Amazon.

AWS today launched Amazon AppFlow, a new integration service that makes it easier for developers to transfer data between AWS and SaaS applications like Google Analytics, Marketo, Salesforce, ServiceNow, Slack, Snowflake and Zendesk. Like similar services, including Microsoft Azure’s Power Automate, for example, developers can trigger these flows based on specific events, at pre-set times or on-demand.

Unlike some of its competitors, though, AWS is positioning this service more as a data transfer service than a way to automate workflows and while the data flow can be bi-directional, AWS’s announcement focuses mostly on moving data from SaaS applications to other AWS services for further analysis. For this, AppFlow also includes a number of tools for transforming the data as it moves through the service.

“Developers spend huge amounts of time writing custom integrations so they can pass data between SaaS applications and AWS services so that it can be analysed; these can be expensive and can often take months to complete,” said AWS principal advocate Martin Beeby in today’s announcement. “If data requirements change, then costly and complicated modifications have to be made to the integrations. Companies that don’t have the luxury of engineering resources might find themselves manually importing and exporting data from applications, which is time-consuming, risks data leakage, and has the potential to introduce human error.”

Every flow (which AWS defines as a call to a source application to transfer data to a destination) costs $0.001 per run, though, in typical AWS fashion, there’s also cost associated with data processing (starting at 0.02 per GB).

“Our customers tell us that they love having the ability to store, process, and analyze their data in AWS. They also use a variety of third-party SaaS applications, and they tell us that it can be difficult to manage the flow of data between AWS and these applications,” said Kurt Kufeld, Vice President, AWS. “Amazon AppFlow provides an intuitive and easy way for customers to combine data from AWS and SaaS applications without moving it across the public Internet. With Amazon AppFlow, our customers bring together and manage petabytes, even exabytes, of data spread across all of their applications – all without having to develop custom connectors or manage underlying API and network connectivity.”

At this point, the number of supported services remains comparatively low, with only 14 possible sources and four destinations (Amazon Redshift and S3, as well as Salesforce and Snowflake). Sometimes, depending on the source you select, the only possible destination is Amazon’s S3 storage service.

Over time, the number of integrations will surely increase, but for now, it feels like there’s still quite a bit more work to do for the AppFlow team to expand the list of supported services.

AWS has long left this market to competitors, even though it has tools like AWS Step Functions for building serverless workflows across AWS services and EventBridge for connections applications. Interestingly, EventBridge currently supports a far wider range of third-party sources, but as the name implies, its focus is more on triggering events in AWS than moving data between applications.

AWS and Facebook today announced two new open-source projects around PyTorch, the popular open-source machine learning framework. The first of these is TorchServe, a model-serving framework for PyTorch that will make it easier for developers to put their models into production. The other is TorchElastic, a library that makes it easier for developers to build fault-tolerant training jobs on Kubernetes clusters, including AWS’s EC2 spot instances and Elastic Kubernetes Service.

In many ways, the two companies are taking what they have learned from running their own machine learning systems at scale and are putting this into the project. For AWS, that’s mostly SageMaker, the company’s machine learning platform, but as Bratin Saha, AWS VP and GM for Machine Learning Services, told me, the work on PyTorch was mostly motivated by requests from the community. And while there are obviously other model servers like TensorFlow Serving and the Multi Model Server available today, Saha argues that it would be hard to optimize those for PyTorch.

“If we tried to take some other model server, we would not be able to quote optimize it as much, as well as create it within the nuances of how PyTorch developers like to see this,” he said. AWS has lots of experience in running its own model servers for SageMaker that can handle multiple frameworks, but the community was asking for a model server that was tailored toward how they work. That also meant adapting the server’s API to what PyTorch developers expect from their framework of choice, for example.

As Saha told me, the server that AWS and Facebook are now launching as open source is similar to what AWS is using internally. “It’s quite close,” he said. “We actually started with what we had internally for one of our model servers and then put it out to the community, worked closely with Facebook, to iterate and get feedback — and then modified it so it’s quite close.”

Bill Jia, Facebook’s VP of AI Infrastructure, also told me, he’s very happy about how his team and the community has pushed PyTorch forward in recent years. “If you look at the entire industry community — a large number of researchers and enterprise users are using AWS,” he said. “And then we figured out if we can collaborate with AWS and push PyTorch together, then Facebook and AWS can get a lot of benefits, but more so, all the users can get a lot of benefits from PyTorch. That’s our reason for why we wanted to collaborate with AWS.”

As for TorchElastic, the focus here is on allowing developers to create training systems that can work on large distributed Kubernetes clusters where you might want to use cheaper spot instances. Those are preemptible, though, so your system has to be able to handle that, while traditionally, machine learning training frameworks often expect a system where the number of instances stays the same throughout the process. That, too, is something AWS originally built for SageMaker. There, it’s fully managed by AWS, though, so developers never have to think about it. For developers who want more control over their dynamic training systems or to stay very close to the metal, TorchElastic now allows them to recreate this experience on their own Kubernetes clusters.

AWS has a bit of a reputation when it comes to open source and its engagement with the open-source community. In this case, though, it’s nice to see AWS lead the way to bring some of its own work on building model servers, for example, to the PyTorch community. In the machine learning ecosystem, that’s very much expected, and Saha stressed that AWS has long engaged with the community as one of the main contributors to MXNet and through its contributions to projects like Jupyter, TensorFlow and libraries like NumPy.

Google, Amazon and Microsoft are the landlords. Amidst the coronavirus economic crisis, startups need a break from paying rent. They’re in a cash crunch. Revenue has stopped flowing in, capital markets like venture debt are hesitant and startups and small-to-medium sized businesses are at risk of either having to lay off huge numbers of employees and/or shut down.

Meanwhile, the tech giants are cash rich. Their success this decade means they’re able to weather the storm for a few months. Their customers cannot.

Cloud infrastructure costs area amongst many startups’ top expense besides payroll. The option to pay these cloud bills later could save some from going out of business or axing huge parts of their staff. Both would hurt the tech industry, the economy and the individuals laid off. But most worryingly for the giants, it could destroy their customer base.

The mass layoffs have already begun. Soon we’re sure to start hearing about sizable companies shutting down, upended by COVID-19. But there’s still an opportunity to stop a larger bloodbath from ensuing.

That’s why I have a proposal: cloud relief.

The platform giants should let startups and small businesses defer their cloud infrastructure payments for three to six months until they can pay them back in installments. Amazon AWS, Google Cloud, Microsoft Azure, these companies’ additional infrastructure products, and other platform providers should let customers pause payment until the worst of the first wave of the COVID-19 economic disruption passes. Profitable SaaS providers like Salesforce could give customers an extension too.

There are plenty of altruistic reasons to do this. They have the resources to help businesses in need. We all need to support each other in these tough times. This could protect tons of families. Some of these startups are providing important services to the public and even discounting them, thereby ramping up their bills while decreasing revenue.

Then there are the PR reasons. After years of techlash and anti-trust scrutiny, here’s the chance for the giants to prove their size can be beneficial to the world. Recruiters could use it as a talking point. “We’re the company that helped save Silicon Valley.” There’s an explanation for them squirreling away so much cash: the rainy day has finally arrived.

But the capitalistic truth and the story they could sell to Wall Street is that it’s not good for our business if our customers go out of business. Look at what happened to infrastructure providers in the dot-com crash. When tons of startups vaporized, so did the profits for those selling them hosting and tools. Any government stimulus for businesses would be better spent by them paying employees than paying the cloud companies that aren’t in danger. Saving one future Netflix from shutting down could cover any short-term loss from helping 100 other businesses.

This isn’t a handout. These startups will still owe the money. They’d just be able to pay it a little later, spread out over their monthly bills for a year or so. Once mass shelter-in-place orders subside, businesses can operate at least a little closer to normal, investors can get less cautious and customers will have the cash they need to pay their dues. Plus interest, if necessary.

Meanwhile, they’ll be locked in and loyal customers for the foreseeable future. Cloud vendors could gate the deferment to only customers that have been with them for X amount of months or that have already spent Y amount on the platform. The vendors also could offer the deferment on the condition that customers add a year or more to their existing contracts. Founders will remember who gave them the benefit of the doubt.

Consider it a marketing expense. Platforms often offer discounts or free trials to new customers. Now it’s existing customers that need a reprieve. Instead of airport ads, the giants could spend the money ensuring they’ll still have plenty of developers building atop them by the end of 2020.

Beyond deferred payment, platforms could just push the due date on all outstanding bills to three or six months from now. Alternatively, they could offer a deep discount such as 50% off for three months if they didn’t want to deal with accruing debt and then servicing it. Customers with multi-year contracts could offered the opportunity to downgrade or renegotiate their contracts without penalties. Any of these might require giving sales quota forgiveness to their account executives.

It would likely be far too complicated and risky to accept equity in lieu of cash, a cut of revenue going forward or to provide loans or credit lines to customers. The clearest and simplest solution is to let startups skip a few payments, then pay more every month later until they clear their debt. When asked for comment or about whether they’re considering payment deferment options, Microsoft declined, and Amazon and Google did not respond.

To be clear, administering payment deferment won’t be simple or free. There are sure to be holes that cloud economists can poke in this proposal, but my goal is to get the conversation started. It could require the giants to change their earnings guidance. Rewriting deals with significantly sized customers will take work on both ends, and there’s a chance of breach of contract disputes. Giants would face the threat of customers recklessly using cloud resources before shutting down or skipping town.

Most taxing would be determining and enforcing the criteria of who’s eligible. The vendors would need to lay out which customers are too big so they don’t accidentally give a cloud-intensive but healthy media company a deferment they don’t need. Businesses that get questionably excluded could make a stink in public. Executing on the plan will require staff when giants are stretched thin trying to handle logistics disruptions, misinformation and accelerating work-from-home usage.

Still, this is the moment when the fortunate need to lend a hand to the vulnerable. Not a hand out, but a hand up. Companies with billions in cash in their coffers could save those struggling to pay salaries. All the fundraisers and info centers and hackathons are great, but this is how the tech giants can live up to their lofty mission statements.

We all live in the cloud now. Don’t evict us. #CloudRelief

—

Thanks to Falon Fatemi, Corey Quinn, Ilya Fushman, Jason Kim, Ilya Sukhar and Michael Campbell for their ideas and feedback on this proposal.

As society comes to grips with the growing worldwide crisis related to the COVID-19 virus, many companies are stepping up in different ways. Today, two major tech companies — Amazon and IBM — each announced programs to encourage developers to find solutions to a variety of problems related to the pandemic.

For starters, AWS, Amazon’s cloud arm, announced the AWS Diagnostic Development Initiative. It has set aside $20 million, which it will distribute in the form of AWS credits and technical support. The program is designed to assist and encourage teams working on COVID-19 diagnostic issues with the goal of developing better diagnostic tooling.

“In our Amazon Web Services (AWS) business, one area where we have heard an urgent need is in the research and development of diagnostics, which consist of rapid, accurate detection and testing of COVID-19. Better diagnostics will help accelerate treatment and containment, and in time, shorten the course of this epidemic,” Teresa Carlson wrote in the company’s Day One blog today.

The program aims to help customers who are working on building diagnostics solutions to bring products to market more quickly, and also encourage teams working on related problems to work together.

The company also announced it was forming an advisory group made up of scientists and health policy experts to assist companies involved with initiative.

Meanwhile, IBM is refocusing its 2020 Call for Code Global Challenge developer contest on not only solving problems related to global climate change, which was this year’s original charter, but also solving issues around the growing virus crisis by building open-source tooling.

“In a very short period of time, COVID-19 has revealed the limits of the systems we take for granted. The 2020 Call for Code Global Challenge will arm you with resources […] to build open source technology solutions that address three main COVID-19 areas: crisis communication during an emergency, ways to improve remote learning, and how to inspire cooperative local communities,” the company wrote in a blog post.

All of these areas are being taxed as more people are forced to stay indoors as we to try to contain the virus. The company hopes to incentivize developers working on these issues to help solve some of these problems.

During a time of extreme social and economic upheaval when all aspects of society are being affected, businesses, academia and governments need to work together to solve the myriad problems related to the virus. These are just a couple of examples of that.

Much like private businesses, the United States government is in the process of moving workloads to the cloud, and facing a similar set of challenges. Today, CircleCI, the continuous delivery developer service, announced a partnership with AWS GovCloud to help federal government entities using AWS’s government platform to modernize their applications development workflows.

“What this means is that it allows us to run our server offering, which is our on-prem offering, and our government customers can run that on dedicated pure cloud resource [on AWS GovCloud],” CircleCI CEO Jim Rose told TechCrunch.

GovCloud is a dedicated, single tenant cloud platform that lets government entities build FedRAMP-compliant secure cloud solutions (other cloud vendors have similar offerings). FedRAMP is a set of government cloud security standards every cloud vendor has to meet to work with the federal government

CircleCI builds modern continuous delivery/continuous integration (CI/CD) pipelines for development teams pushing changes to the application in a rapid change cycle.

“What GovCloud allows us to do is now provide that same level of security and service for government customers that wanted us to do so in an on prem environment in a dedicated single tenant environment [in the cloud],” Rose explained.

While there are a number of steps involved in building cloud applications, Rose said they are sticking to their core strength around building continuous delivery pipelines. As he says, if you have a legacy mainframe application that changes once every year or two, using CircleCI wouldn’t make sense, but as you begin to modernize, that’s where his company could help.

“[CircleCi comes into play] when you get into more modern cloud applications that are changing in some cases hundreds of times a day, and the sources of change for those applications is getting really diverse and managing that is becoming more complex,” Rose said.

This partnership could involve working directly with an agency, as it has done with the Small Business Administration (SBA), or it might involve a systems integrator, or even AWS, inviting them to be part of a larger RFP.

Rose says he realizes that working with the government can sometimes be controversial. Companies from Chef to Salesforce to Google have run afoul with employees who don’t want to work with certain agencies like DoD or ICE. He says his company has tended to focus on areas where agencies are looking to improve citizen interactions and steered away from other areas.

“From our perspective, given that we’re not super involved in a lot of those areas, but we want to get in front of it, both commercially, as well as on the government side, and determine what falls within the fence line and what’s outside of it,” he said.

Today, AWS made public its Motion to Supplement the Record in its protest of the JEDI contract decision. As part of that process, the company has announced it wants to depose President Trump and Secretary of Defense Mark Esper.

When Amazon announced at the end of last year that it was protesting the DoD’s decision to award the $10 billion, decade-long JEDI contract to Microsoft, the company made clear that it was not happy with the decision. The company believes that the president steered the contract away from Amazon because of personal political differences with Amazon CEO Jeff Bezos, who also owns The Washington Post.

“President Trump has repeatedly demonstrated his willingness to use his position as President and Commander in Chief to interfere with government functions – including federal procurements – to advance his personal agenda. The preservation of public confidence in the nation’s procurement process requires discovery and supplementation of the administrative record, particularly in light of President Trump’s order to ‘screw Amazon.’ The question is whether the President of the United States should be allowed to use the budget of the DoD to pursue his own personal and political ends,” an AWS spokesperson said in a statement.

This is consistent with public statements the company has been making since the DoD made the surprise decision in October to go with Microsoft. It had been widely believed that Amazon would win the contract, and there was much wrangling and complaining throughout the procurement process that the contract had been designed to favor Amazon, something that the DoD repeatedly denied.

At AWS re:Invent at the end last year, AWS CEO Andy Jassy made it clear he was unhappy with the decision and that he believed the president showed bias. “I think that we ended up with a situation where there was political interference. When you have a sitting president, who has shared openly his disdain for a company, and the leader of that company, it makes it really difficult for government agencies, including the DoD, to make objective decisions without fear of reprisal,” Jassy said last year.

Sources say that the DoD gave Amazon a written debriefing after the decision to award the contract to Microsoft, but the company is particularly upset that the department has failed to respond in a timely fashion to requests for additional information and questions, as required by law.