Sep
23
2020
--

WhyLabs brings more transparancy to ML ops

WhyLabs, a new machine learning startup that was spun out of the Allen Institute, is coming out of stealth today. Founded by a group of former Amazon machine learning engineers, Alessya Visnjic, Sam Gracie and Andy Dang, together with Madrona Venture Group principal Maria Karaivanova, WhyLabs’ focus is on ML operations after models have been trained — not on building those models from the ground up.

The team also today announced that it has raised a $4 million seed funding round from Madrona Venture Group, Bezos Expeditions, Defy Partners and Ascend VC.

Visnjic, the company’s CEO, used to work on Amazon’s demand forecasting model.

“The team was all research scientists, and I was the only engineer who had kind of tier-one operating experience,” she told me. “So I thought, “Okay, how bad could it be? I carried the pager for the retail website before. But it was one of the first AI deployments that we’d done at Amazon at scale. The pager duty was extra fun because there were no real tools. So when things would go wrong — like we’d order way too many black socks out of the blue — it was a lot of manual effort to figure out why issues were happening.”

Image Credits: WhyLabs

But while large companies like Amazon have built their own internal tools to help their data scientists and AI practitioners operate their AI systems, most enterprises continue to struggle with this — and a lot of AI projects simply fail and never make it into production. “We believe that one of the big reasons that happens is because of the operating process that remains super manual,” Visnjic said. “So at WhyLabs, we’re building the tools to address that — specifically to monitor and track data quality and alert — you can think of it as Datadog for AI applications.”

The team has brought ambitions, but to get started, it is focusing on observability. The team is building — and open-sourcing — a new tool for continuously logging what’s happening in the AI system, using a low-overhead agent. That platform-agnostic system, dubbed WhyLogs, is meant to help practitioners understand the data that moves through the AI/ML pipeline.

For a lot of businesses, Visnjic noted, the amount of data that flows through these systems is so large that it doesn’t make sense for them to keep “lots of big haystacks with possibly some needles in there for some investigation to come in the future.” So what they do instead is just discard all of this. With its data logging solution, WhyLabs aims to give these companies the tools to investigate their data and find issues right at the start of the pipeline.

Image Credits: WhyLabs

According to Karaivanova, the company doesn’t have paying customers yet, but it is working on a number of proofs of concepts. Among those users is Zulily, which is also a design partner for the company. The company is going after mid-size enterprises for the time being, but as Karaivanova noted, to hit the sweet spot for the company, a customer needs to have an established data science team with 10 to 15 ML practitioners. While the team is still figuring out its pricing model, it’ll likely be a volume-based approach, Karaivanova said.

“We love to invest in great founding teams who have built solutions at scale inside cutting-edge companies, who can then bring products to the broader market at the right time. The WhyLabs team are practitioners building for practitioners. They have intimate, first-hand knowledge of the challenges facing AI builders from their years at Amazon and are putting that experience and insight to work for their customers,” said Tim Porter, managing director at Madrona. “We couldn’t be more excited to invest in WhyLabs and partner with them to bring cross-platform model reliability and observability to this exploding category of MLOps.”

Sep
08
2020
--

The $10B JEDI contract is locked, loaded and still completely stuck

The other day I took a moment to count the number of stories we’ve done on TechCrunch on the DoD’s $10 billion, decade-long, winner-take-all, JEDI cloud contract. This marks the 30th time we’ve written about this deal over the last two years, and it comes after a busy week last week in JEDI cloud contract news.

That we’re still writing about this is fairly odd if you consider the winner was announced last October when the DoD chose Microsoft, but there is no end in sight to the on-going drama that is this procurement process.

Government contracts don’t typically catch our attention at TechCrunch, but this one felt different early on. There was the size and scope of the deal of course. There was the cute play on the “Star Wars” theme. There was Oracle acting like a batter complaining to the umpire before the first pitch was thrown. There was the fact that everyone thought Amazon would win until it didn’t.

There was a lot going on. In fact, there’s still a lot going on with this story.

Oracle doth protest too much

Let’s start with Oracle, which dispatched CEO Safra Catz to the White House in April 2018 even before the RFP had been written. She was setting the stage to complain that the deal was going to be set up to favor Amazon, something that Oracle alleged until the day Microsoft was picked the winner.

Catz had been on the Trump transition team and so had the ear of the president. While the president certainly interjected himself in this process, it’s not known how much influence that particular meeting might have had. Suffice to say that it was only the first volley in Oracle’s long war against the JEDI contract procurement process.

It would include official complaints with the Government Accountability Office and a federal lawsuit worth not coincidentally $10 billion. It would claim the contract favored Amazon. It would argue that the one-vendor approach wasn’t proper. It would suggest that because the DoD had some former Amazon employees helping write the RFP, that it somehow favored Amazon. The GAO and two court cases found otherwise, ruling against Oracle every single time.

It’s worth noting that the Court of Appeals ruling last week indicated that Oracle didn’t even meet some of the basic contractual requirements, all the while complaining about the process itself from the start.

Amazon continues to press protests

Nobody was more surprised that Amazon lost the deal than Amazon itself. It still believes to this day that it is technically superior to Microsoft and that it can offer the DoD the best approach. The DoD doesn’t agree. On Friday, it reaffirmed its choice of Microsoft. But that is not the end of this, not by a long shot.

Amazon has maintained since the decision was made last October that the decision-making process had been tainted by presidential interference in the process. They believe that because of the president’s personal dislike of Amazon CEO Jeff Bezos, who also owns the Washington Post, he inserted himself in the process to prevent Bezos’ company from winning that deal.

In January, Amazon filed a motion to stop work on the project until this could all be sorted out. In February, a judge halted work on the project until Amazon’s complaints could be heard by the court. It is September and that order is still in place.

In a blog post on Friday, Amazon reiterated its case, which is based on presidential interference and what it believes is technical superiority. “In February, the Court of Federal Claims stopped performance on JEDI. The Court determined AWS’s protest had merit, and that Microsoft’s proposal likely failed to meet a key solicitation requirement and was likely deficient and ineligible for award. Our protest detailed how pervasive these errors were (impacting all six technical evaluation factors), and the Judge stopped the DoD from moving forward because the very first issue she reviewed demonstrated serious flaws,” Amazon wrote in the post.

Microsoft for the win?

Microsoft on the other hand went quietly about its business throughout this process. It announced Azure Stack, a kind of portable cloud that would work well as a field operations computer system. It beefed up its government security credentials.

Even though Microsoft didn’t agree with the one-vendor approach, indicating that the government would benefit more from the multivendor approach many of its customers were taking, it made clear if those were the rules, it was in it to win it — and win it did, much to the surprise of everyone, especially Amazon.

Yet here we are, almost a year later and in spite of the fact that the DoD found once again, after further review, that Microsoft is still the winner, the contract remains in limbo. Until that pending court case is resolved, we will continue to watch and wait and wonder if this will ever be truly over, and the JEDI cloud contract will actually be implemented.

May
08
2020
--

Microsoft and AWS exchange poisoned pen blog posts in latest Pentagon JEDI contract spat

Microsoft and Amazon are at it again as the fight for the Defense Department JEDI contract continues. In a recent series of increasingly acerbic pronouncements, the two companies continue their ongoing spat over the $10 billion, decade-long JEDI contract spoils.

As you may recall (or not), last fall in a surprise move, the DoD selected Microsoft as the winning vendor in the JEDI winner-take-all cloud infrastructure sweepstakes. The presumed winner was always AWS, but when the answer finally came down, it was not them.

To make a very long story short, AWS took exception to the decision and went to court to fight it. Later it was granted a stay of JEDI activities between Microsoft and the DoD, which as you can imagine did not please Microsoft . Since then, the two companies have been battling in PR pronouncements and blog posts trying to get the upper hand in the war for public opinion.

That fight took a hard turn this week when the two companies really went at it in dueling blog posts after Amazon filed its latest protest.

First there was Microsoft with PR exec Frank Shaw taking exception to AWS’s machinations, claiming the company just wants a do-over:

This latest filing – filed with the DoD this time – is another example of Amazon trying to bog down JEDI in complaints, litigation and other delays designed to force a do-over to rescue its failed bid.

Amazon’s Drew Herdner countered in a blog post published this morning:

Recently, Microsoft has published multiple self-righteous and pontificating blog posts that amount to nothing more than misleading noise intended to distract those following the protest.

The bottom line is that Microsoft believes it won the contract fair and square with a more competitive bid, while Amazon believes it should have won on technical superiority, and that there was political interference from the president because he doesn’t like Amazon CEO Jeff Bezos, who also owns the Washington Post.

If you’ve been following this story from the beginning (as I have), you know it has taken a series of twists and turns. It’s had lawsuits, complaints, drama and intrigue. The president has inserted himself into it, too. There have been accusations of conflicts of interest. There have been investigations, lawsuits and more investigations.

Government procurement tends to be pretty bland, but from the start when the DoD chose to use the cutesy Star Wars-driven acronym for this project, it has been anything but. Now it’s come down to two of the world’s largest tech companies exchanging angry blog posts. Sooner or later this is going to end right?

Apr
15
2020
--

DoD Inspector General report finds everything was basically hunky-dory with JEDI cloud contract bid

While controversy has dogged the $10 billion, decade-long JEDI contract since its earliest days, a report by the DoD’s Inspector General’s Office concluded today that, while there were some funky bits and potential conflicts, overall the contract procurement process was fair and legal and  the president did not unduly influence the process in spite of public comments.

There were a number of issues along the way about whether the single contractor award was fair or reasonable, about whether there were was White House influence on the decision, and whether the president wanted to prevent Amazon founder Jeff Bezos, who also owns the Washington Post, from getting the contract.

There were questions about whether certain personnel, who had been or were about to be Amazon employees, had undue influence on the contents of the RFP or if former Secretary of Defense showed favor to Amazon, which ultimately did not even win the contract, and that one of Mattis’ under secretaries, in fact, owned stock in Microsoft .

It’s worth noting that the report states clearly that it is not looking at the merits of this contract award or whether the correct company won on technical acumen. It was looking at all of these controversial parts that came up throughout the process. As the report stated:

“In this report, we do not draw a conclusion regarding whether the DoD appropriately awarded the JEDI Cloud contract to Microsoft rather than Amazon Web Services. We did not assess the merits of the contractors’ proposals or DoD’s technical or price evaluations; rather we reviewed the source selection process and determined that it was in compliance with applicable statutes, policies, and the evaluation process described in the Request for Proposals.”

Although the report indicates that the White House would not cooperate with the investigation into potential bias, the investigators claim they had enough discussions with parties involved with the decision to conclude that there was no undue influence on the White House’s part:

“However, we believe the evidence we received showed that the DoD personnel who evaluated the contract proposals and awarded Microsoft the JEDI Cloud contract were not pressured regarding their decision on the award of the contract by any DoD leaders more senior to them, who may have communicated with the White House,” the report stated.

The report chose to blame the media instead, at least for partly giving the impression that the White House had influenced the process, stating:

“Yet, these media reports, and the reports of President Trump’s statements about Amazon, ongoing bid protests and “lobbying” by JEDI Cloud competitors, as well as inaccurate media reports about the JEDI Cloud procurement process, may have created the appearance or perception that the contract award process was not fair or unbiased.”

It’s worth noting that we reported that AWS president Andy Jassy made it clear in a press conference at AWS re:Invent in December that the company believed the president’s words had influenced the process.

“I think that we ended up with a situation where there was political interference. When you have a sitting president, who has shared openly his disdain for a company, and the leader of that company, it makes it really difficult for government agencies, including the DoD, to make objective decisions without fear of reprisal.”

As for other points of controversy, such as those previously referenced biases, all were found lacking by the Inspector General. While the earliest complaints from Oracle and others were that Deap Ubhi and Victor Gavin, two individuals involved in drafting the RFP, failed to disclose they were offered jobs by Amazon during that time.

The report concluded that while Ubhi violated ethics rules, his involvement wasn’t substantial enough to influence the RFP (which again, Amazon didn’t win). “However, we concluded that Mr. Ubhi’s brief early involvement in the JEDI Cloud Initiative was not substantial and did not provide any advantage to his prospective employer, Amazon…,” the report stated.

The report found Gavin did not violate any ethics rules in spite of taking a job with Amazon because he had disqualified himself from the process, nor did the report find that former Secretary Mattis had any ethical violations in its investigation.

One final note: Stacy Cummings, Principal Deputy Assistant Secretary of Defense for Acquisition and Deputy Assistant Secretary of Defense for Acquisition Enablers, who worked for Mattis, owned some stock in Microsoft and did not disclose this. While the report found that was a violation of ethics guidelines, it ultimately concluded this did not unduly influence the award to Microsoft.

While the report is a substantial, 313 pages, it basically concludes that as far as the purview of the Inspector General is concerned, the process was basically conducted in a fair way. The court case, however involving Amazon’s protest of the award to Microsoft continues. And the project remains on hold until that is concluded.

Note: Microsoft and Amazon did not respond to requests from TechCrunch for comments before we published this article. If that changes, we will update accordingly.

Report on the Joint Enterprise Defense Infrastructure (Jedi) Cloud Procurement Dodig-2020-079 by TechCrunch on Scribd

Mar
19
2020
--

Even in the age of COVID-19, you need to stay focused on the customer

It’s easy to think, as we find ourselves in the midst of a truly unprecedented situation, that the rules of building a successful business have suddenly changed. While the world may be topsy-turvy at the moment, keeping your customer at the center of your business strategy is more important than ever.

That means finding creative ways to engage with your customers and thinking deeply about what they need as the world changes before our eyes.

As a small example on a local level, Pandemonium Books and Games in Cambridge, Mass. has started offering same-day delivery to neighborhoods in the Boston area for a $5 fee and a $20 minimum purchase.

This is taking a difficult situation and finding a way to stay connected with customers, while keeping the business going through difficult times. It’s something that your most loyal customers will certainly remember when we return to some semblance of normalcy — and it’s just a great community service.

When you hear from leaders of the world’s most successful technology companies, whether it’s Jeff Bezos at Amazon or Marc Benioff at Salesforce, these two executives are constantly pushing their organizations to put the customer first.

At Amazon, that manifests itself in the company motto that it’s always Day 1. That motto means they never can become complacent and always place the customer first. In his 2016 Letter to Shareholders, Bezos described what he meant:

There are many ways to center a business. You can be competitor focused, you can be product focused, you can be technology focused, you can be business model focused, and there are more. But in my view, obsessive customer focus is by far the most protective of Day 1 vitality.

Benioff runs his company with a similar world view, and it’s no coincidence that both companies are so wildly successful. In his recent book, Trailblazer, Benioff wrote about the importance of relentless customer focus:

Nothing a company does is more essential than how it engages with customers. In a world where online portals are replacing customer service centers and algorithms are replacing humans on the front lines, companies like ours continually need to show that the personal connections our customers craved were still — and always would be — there.

In our current crisis, that focus becomes ever more important and universal. In his last interview before his death in January, Clayton Christensen, author of the seminal book Innovator’s Dilemma, told MIT Sloan Management Review that while these organizations had other things going for them, customer centricity was certainly a big factor in their success:

They have all built organizations that have put the customers, and their Job to Be Done, at the center. They also have demonstrated the ability to manage emergent strategy well. However, they also have been in the fortunate circumstance where their core businesses have been growing at phenomenal rates, and they have had the presence of the founder to help, to personally get involved in key strategic decisions.

While you don’t want to appear like you are taking advantage of a bad situation, there are ways you can help your customers by thinking of new ways engage and help them in a difficult time. Many companies are offering services for free for the next several months to help customers get through the financial uncertainty we are facing in the near term. Others are posting free content and access to other resources on websites.

While it’s understood that some customers simply won’t have money to spend in the coming months, those that do will have different needs than they did before and you have to be ready to address them, whatever that means to your business.

This virus is going to force us to rethink about a lot of the ways we run our businesses, our society and our lives, but if you keep your customer at the center of all your decisions, even in the midst of such a crisis, you will be setting the foundation for a successful business whenever we return to normal.

Mar
13
2020
--

Pentagon asks court for time to reconsider JEDI award to Microsoft

The JEDI contract award process might never be done. Following legal challenges from Amazon after the Pentagon’s massive, $10 billion cloud contract was awarded to Microsoft in October, the Pentagon indicated in court documents last night that it wishes to reconsider the award.

It’s just the latest plot twist in an epic government procurement saga.

Here’s what we know. The Pentagon filing is based on Amazon’s complaints about the technical part of the deal only. Amazon has said that it believes political interference influenced the awarding of the contract. However, the cloud computing giant also believes it beat Microsoft on the technical merits in a majority of instances required in the request for proposals issued by the Pentagon.

In fact, sources told TechCrunch, “AWS’s protest identified evaluation errors, clear deficiencies and unmistakable bias in six of the eight evaluation factors.”

Obviously Amazon was happy to hear this news. “We are pleased that the DoD has acknowledged ‘substantial and legitimate’ issues that affected the JEDI award decision, and that corrective action is necessary,” a spokesperson stated.

“We look forward to complete, fair, and effective corrective action that fully insulates the re-evaluation from political influence and corrects the many issues affecting the initial flawed award.”

As would expect, Microsoft thinks that the DoD made the correct choice, and believes the review will bear that out. “Over two years, the DoD reviewed dozens of factors and sub factors and found Microsoft equal or superior to AWS on every factor. We remain confident that Microsoft’s proposal was technologically superior, continues to offer the best value, and is the right choice for the DoD,” Microsoft VP of communications Frank Shaw said.

The court granted the Pentagon 120 days to review the results again, but indicated it could take longer. In the meantime, the project is at a standstill.

On Friday, the court issued a ruling that Amazon was likely to succeed on its complaint on merit, and that could have been the impetus of this latest action by the Pentagon.

While the political influence piece might not be overtly part of this filing, it does lurk in the background. The president has made it clear that he doesn’t like Amazon founder and CEO Jeff Bezos, who also owns The Washington Post. As we wrote last year:

Amazon, for instance, could point to Jim Mattis’ book where he wrote that the president told the then Defense Secretary to “screw Bezos out of that $10 billion contract.” Mattis says he refused, saying he would go by the book, but it certainly leaves the door open to a conflict question.

As we previously reported, AWS CEO Andy Jassy stated at a press event at AWS re:Invent in December that the company believed there was political bias at play in the decision-making process.

“What I would say is that it’s fairly obvious that we feel pretty strongly that it was not adjudicated fairly,” he said. He added, “I think that we ended up with a situation where there was political interference. When you have a sitting president, who has shared openly his disdain for a company, and the leader of that company, it makes it really difficult for government agencies, including the DoD, to make objective decisions without fear of reprisal.”

The story has been updated with a comment from Microsoft. We have requested comment from DoD and will update the story should they respond.

Aug
01
2019
--

President throws latest wrench in $10B JEDI cloud contract selection process

The $10 billion, decade-long JEDI cloud contract drama continues. It’s a process that has been dogged by complaints, regulatory oversight and court cases. Throughout the months-long selection process, the Pentagon has repeatedly denied accusations that the contract was somehow written to make Amazon a favored vendor, but today The Washington Post reports President Trump has asked the newly appointed Defense Secretary, Mark T. Esper, to examine the process because of concerns over that very matter.

The Defense Department called for bids last year for a $10 billion, decade-long contract. From the beginning, Oracle in particular complained that the process favored Amazon. Even before the RFP process began Oracle executive Safra Catz took her concerns directly to the president, but at that time he did not intervene. Later, the company filed a complaint with the Government Accountability Office, which ruled that the procurement process was fair.

Finally, the company took the case to court, alleging that a person involved in defining the selection process had a conflict of interest, due to being an employee at Amazon before joining the DoD. That case was dismissed last month.

In April, the DoD named Microsoft and Amazon as the two finalists, and the winner was finally expected to be named some time this month. It appeared that we were close to the finish line, but now that the president has intervened at the 11th hour, it’s impossible to know what the outcome will be.

What we do know is that this is a pivotal project for the DoD, which is aimed at modernizing the U.S. military for the next decade and beyond. The fact is that the two finalists made perfect sense. They are the two market leaders, and each has tools, technologies and experience working with sensitive government contracts.

Amazon is the market leader, with 33% market share. Microsoft is No. 2, with 16%. The No. 3 vendor, Google, dropped out before the RFP process began. It is unclear at this point whether the president’s intervention will have any influence on the final decision, but The Washington Post reports it is an unusual departure from government procurement procedures.

Oct
15
2018
--

Jeff Bezos is just fine taking the Pentagon’s $10B JEDI cloud contract

Some tech companies might have a problem taking money from the Department of Defense, but Amazon isn’t one of them, as CEO Jeff Bezos made clear today at the Wired25 conference. Just last week, Google pulled out of the running for the Pentagon’s $10 billion, 10-year JEDI cloud contract, but Bezos suggested that he was happy to take the government’s money.

Bezos has been surprisingly quiet about the contract up until now, but his company has certainly attracted plenty of attention from the companies competing for the JEDI deal. Just last week IBM filed a formal protest with the Government Accountability Office claiming that the contract was stacked in favor one vendor. And while it didn’t name it directly, the clear implication was that company was the one owned by Bezos.

Last summer Oracle also filed a protest and also complained that they believed the government had set up the contract to favor Amazon, a charge spokesperson Heather Babb denied. “The JEDI Cloud final RFP reflects the unique and critical needs of DOD, employing the best practices of competitive pricing and security. No vendors have been pre-selected,” she said last month.

While competitors are clearly worried about Amazon, which has a substantial lead in the cloud infrastructure market, the company itself has kept quiet on the deal until now. Bezos set his company’s support in patriotic terms and one of leadership.

“Sometimes one of the jobs of the senior leadership team is to make the right decision, even when it’s unpopular. And if if big tech companies are going to turn their back on the US Department of Defense, this country is going to be in trouble,” he said.

“I know everyone is conflicted about the current politics in this country, but this country is a gem,” he added.

While Google tried to frame its decision as taking a principled stand against misuse of technology by the government, Bezos chose another tack, stating that all technology can be used for good or ill. “Technologies are always two-sided. You know there are ways they can be misused as well as used, and this isn’t new,” Bezos told Wired25.

He’s not wrong of course, but it’s hard not to look at the size of the contract and see it as purely a business decision on his part. Amazon is as hot for that $10 billion contract as any of its competitors. What’s different in this talk is that Bezos made it sound like a purely patriotic decision, rather than economic one.

The Pentagon’s JEDI contract could have a value of up to $10 billion with a maximum length of 10 years. The contract is framed as a two year deal with two three-year options and a final one for two years. The DOD can opt out before exercising any of the options.

Bidding for the contract closed last Friday. The DOD is expected to choose the winning vendor next April.

Apr
15
2017
--

Your customer’s online journey is only part of the story

 For years we’ve been hearing about the promise of customer experience management, the notion that you can understand your customer at a highly detailed level and serve them the content, products, services and even ads that matter (or at least make sense) to them.
Today’s marketing software is bringing that idea closer to reality.
But technology is only part of the brand-consumer… Read More

Apr
28
2016
--

Amazon’s Web Services are shining in its latest earnings beat

amazon Amazon posted its first-quarter earnings today, and boy did they not disappoint: it beat what analysts were expecting on nearly all fronts, and the stock is up more than 12% after its huge beat. The big one here, in particular, is Amazon Web Services. AWS has become a go-to for most businesses, so it’s not surprising that it’s seeing that segment continue to grow steadily. The… Read More

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com