Jan
05
2021
--

How Segment redesigned its core systems to solve an existential scaling crisis

Segment, the startup Twilio bought last fall for $3.2 billion, was just beginning to take off in 2015 when it ran into a scaling problem: It was growing so quickly, the tools it had built to process marketing data on its platform were starting to outgrow the original system design.

Inaction would cause the company to hit a technology wall, managers feared. Every early-stage startup craves growth and Segment was no exception, but it also needed to begin thinking about how to make its data platform more resilient or reach a point where it could no longer handle the data it was moving through the system. It was — in a real sense — an existential crisis for the young business.

The project that came out of their efforts was called Centrifuge, and its purpose was to move data through Segment’s data pipes to wherever customers needed it quickly and efficiently at the lowest operating cost.

Segment’s engineering team began thinking hard about what a more robust and scalable system would look like. As it turned out, their vision would evolve in a number of ways between the end of 2015 and today, and with each iteration, they would take a leap in terms of how efficiently they allocated resources and processed data moving through its systems.

The project that came out of their efforts was called Centrifuge, and its purpose was to move data through Segment’s data pipes to wherever customers needed it quickly and efficiently at the lowest operating cost. This is the story of how that system came together.

Growing pains

The systemic issues became apparent the way they often do — when customers began complaining. When Tido Carriero, Segment’s chief product development officer, came on board at the end of 2015, he was charged with finding a solution. The issue involved the original system design, which like many early iterations from startups was designed to get the product to market with little thought given to future growth and the technical debt payment was coming due.

“We had [designed] our initial integrations architecture in a way that just wasn’t scalable in a number of different ways. We had been experiencing massive growth, and our CEO [Peter Reinhardt] came to me maybe three times within a month and reported various scaling challenges that either customers or partners of ours had alerted him to,” said Carriero.

The good news was that it was attracting customers and partners to the platform at a rapid clip, but it could all have come crashing down if the company didn’t improve the underlying system architecture to support the robust growth. As Carriero reports, that made it a stressful time, but having come from Dropbox, he was actually in a position to understand that it’s possible to completely rearchitect the business’s technology platform and live to tell about it.

“One of the things I learned from my past life [at Dropbox] is when you have a problem that’s just so core to your business, at a certain point you start to realize that you are the only company in the world kind of experiencing this problem at this kind of scale,” he said. For Dropbox that was related to storage, and for Segment it was processing large amounts of data concurrently.

In the build-versus-buy equation, Carriero knew that he had to build his way out of the problem. There was nothing out there that could solve Segment’s unique scaling issues. “Obviously that led us to believe that we really need to think about this a little bit differently, and that was when our Centrifuge V2 architecture was born,” he said.

Building the imperfect beast

The company began measuring system performance, at the time processing 8,442 events per second. When it began building V2 of its architecture, that number had grown to an average of 18,907 events per second.

Nov
03
2020
--

How startups can shake up their first idea and still crush the market

When Quibi announced it was shutting its doors recently after raising $1.75 billion, it begged an obvious question: If the original idea didn’t work, why not adjust its model or do something completely different while it still had capital? It wouldn’t have been the first company to decide to shift gears. Perhaps because of the unusually large amount of money it burned through in just six months of public operation, pivoting wasn’t an option for Quibi, but it has been for countless other successful companies over the years. Sometimes an original idea simply doesn’t pan out, a market gets too crowded or a company’s founders stumble onto something they have built that is actually a better business than the original idea.

There are many such examples:

These examples — and many more — show that when your first approach doesn’t work, pivoting may be the the only logical course, but it takes courage from founders and patience from investors.

We spoke to several founders and VCs who have been through this to find out how pivots happen, and how all the parties involved adjust to shifting priorities.

Sometimes it’s a long and twisting road

A big part of founding a company is having vision. You need to believe in your idea of course, but that doesn’t mean it’s the right way to go. Sometimes it pays to move on. The king of pivots might be the aptly named Pivotal, which changed direction several times and even swapped owners before it went public and got acquired, all in the span of about 20 years. Ed Sim, co-founder at boldstart ventures was part of Dawntreader Ventures in the late 90s when his firm invested in an early version of the company called Metapa. Sim had a front row seat to every twist and turn in the company’s long and intricate history.

“Greenplum, which was sold to EMC and eventually became Pivotal Software, was initially called Metapa. Metapa was in the Akamai space and as the markets cratered in 2001 for funding infrastructure projects, Scott Yara (the company’s founder) and team bought a small company called Didera and turned it into Greenplum, the first petabyte scale data warehouse built on top of open-source technology,” Sim told TechCrunch. It didn’t end there though as Sim continued, “Once again, years later, Scott recruited his replacement CEO, Bill Cook, and they paired together to sell Greenplum to EMC and eventually spin back out and take the company public as Pivotal Software.

It’s worth noting that Pivotal eventually ran into financial problems when its stock tanked last year, but fellow Dell/EMC family member VMware saved the day by acquiring it for $2.7 billion.

Sometimes you stumble onto an idea

Segment, the customer-data platform company that was recently sold to Twilio for $3.2 billion was originally a college lecture sentiment platform, according to CEO and co-founder Peter Reinhardt. “Our first idea was a classroom lecture tool, ClassMetric, which gave students a button they could press in class to let professors know, in real-time, that they were confused. I like to think of it like a pulse monitor for class confusion,” Reinhardt told TechCrunch

That idea quickly failed when professors testing it found that inviting students to open their laptops to test their sentiment just led them to start playing Solitaire or checking Facebook. Professors weren’t thrilled and they moved on. The founders, who were MIT students at the time, decided they wanted to build an analytics tool instead, but it turned out that competition from Google Analytics and Mixpanel at the time proved too steep.

“We spent a year on development, but it was a crowded market and we struggled to carve out our own niche. We were rapidly running out of capital and the pressure was on to find something new,” he said. They were actually considering simply packing it in, but they had developed a tiny open-source tool called analytics.js, which they used to get data into their failed analytics product. At that point, desperate for an idea, one of the founders suggested posting the open-source tool on Hacker News.

Nov
02
2020
--

Twilio wraps $3.2B purchase of Segment after warp-speed courtship

It was barely a month ago we began hearing rumors that Twilio was interested in acquiring Segment. The $3.2 billion deal was officially announced three weeks ago, and this morning the communications API company announced that the deal had closed, astonishingly fast for an acquisition of this size.

While we can’t know for sure, the speed with which the deal closed could suggest that it was in the works longer than we had known, and when we began hearing rumors of the acquisition, it could have already been signed, sealed and delivered. In addition, the fact that Twilio CEO Jeff Lawson and Segment CEO Peter Reinhardt knew one another before coming to terms might have helped accelerate the process.

Regardless, the two companies are a nice fit. Both deal with the API economy, providing a set of tools to help developers easily add a particular set of functions to their applications. For Twilio, that’s a set of communications APIs, while Segment focuses on customer data.

When you pull the two sets of tooling together, and combine that with Twilio’s 2018 SendGrid acquisition, you can see the possibility to build more complete applications for interacting with customers at every level, including basic communications like video, SMS and audio from Twilio, as well as customer data from Segment and customized emails and ads based on those interactions from SendGrid.

As companies increasingly focus on digital engagement, especially in the midst of a pandemic, Twilio’s Lawson believes the biggest roadblock to this type of engagement has been that data has been locked in silos, precisely the kind of problem that Segment has been attacking.

“With the addition of Segment, Twilio’s Customer Engagement Platform now enables companies to both understand their customer and engage with them digitally — the combination is key to building great digital experiences,” Lawson said in a statement.

In a recent post looking at the reasoning behind the deal, Brent Leary, founder and principal analyst at CRM Essentials, saw it this way: “This move allows Twilio to impact the data-insight-interaction-experience transformation process by removing friction from developers using their platform,” Leary explained.

With the deal closed, Segment will become a division of Twilio. Reinhardt will continue to be CEO, and will report directly to Lawson.

Oct
12
2020
--

Twilio’s $3.2B Segment acquisition is about helping developers build data-fueled apps

The pandemic has forced businesses to change the way they interact with customers. Whether it’s how they deliver goods and services, or how they communicate, there is one common denominator, and that’s that everything is being forced to be digitally driven much faster.

To some extent, that’s what drove Twilio to acquire Segment for $3.2 billion today. (We wrote about the deal over the weekend. Forbes broke the story last Friday night.) When you get down to it, the two companies fit together well, and expand the platform by giving Twilio customers access to valuable customer data. Chee Chew, Twilio’s chief product officer, says while it may feel like the company is pivoting in the direction of customer experience, they don’t necessarily see it that way.

“A lot of people have thought about us as a communications company, but we think of ourselves as a customer engagement company. We really think about how we help businesses communicate more effectively with their customers,” Chew told TechCrunch.

Laurie McCabe, co-founder and partner at SMB Group, sees the move related to the pandemic and the need companies have to serve customers in a more fully digital way. “More customers are realizing that delivering a great customer experience is key to survive through the pandemic, and thriving as the economy recovers — and are willing to spend to do this even in uncertain times,” McCabe said.

Certainly Chew recognized that Segment gives them something they were lacking by providing developers with direct access to customer data, and that could lead to some interesting applications.

“The data capabilities that Segment has are providing a full view of the customer. It really layers across everything we do. I think of it as a horizontal add across the channels and extending beyond. So I think it really helps us advance in a different sort of way […] towards getting the holistic view of the customer and enabling our customers to build intelligence services on top,” he said.

Brent Leary, founder and principal analyst at CRM Essentials, sees Segment helping to provide a powerful data-fueled developer experience. “This move allows Twilio to impact the data-insight-interaction-experience transformation process by removing friction from developers using their platform,” Leary explained. In other words, it gives developers that ability that Chew alluded to, to use data to build more varied applications using Twilio APIs.

Paul Greenberg, author of CRM at the Speed of Light, and founder and principal analyst at 56 Group, agrees, saying, “Segment gives Twilio the ability to use customer data in what is already a powerful unified communications platform and hub. And since it is, in effect, APIs for both, the flexibility [for developers] is enormous,” he said.

That may be so, but Holger Mueller, an analyst at Constellation Research, says the company has to be seeing that the pure communication parts of the platform like SMS are becoming increasingly commoditized, and this deal, along with the SendGrid acquisition in 2018, gives Twilio a place to expand its platform into a much more lucrative data space.

“Twilio needs more growth path and it looks like its strategy is moving up the stack, at least with the acquisition of Segment. Data movement and data residence compliance is a huge headache for enterprises when they build their next generation applications,” Mueller said.

As Chew said, early on the problems were related to building SMS messages into applications and that was the problem that Twilio was trying to solve because that’s what developers needed at the time, but as it moves forward, it wants to provide a more unified customer communications experience, and Segment should help advance that capability in a big way for them.

Oct
11
2020
--

Twilio is buying customer data startup Segment for between $3B and $4B

Sources have told TechCrunch that Twilio intends to acquire customer data startup Segment for between $3 and $4 billion. Forbes broke the story on Friday night, reporting a price tag of $3.2 billion.

We have heard from a couple of industry sources that the deal is in the works and could be announced as early as Monday.

Twilio and Segment are both API companies. That means they create an easy way for developers to tap into a specific type of functionality without writing a lot of code. As I wrote in a 2017 article on Segment, it provides a set of APIs to pull together customer data from a variety of sources:

Segment has made a name for itself by providing a set of APIs that enable it to gather data about a customer from a variety of sources like your CRM tool, customer service application and website and pull that all together into a single view of the customer, something that is the goal of every company in the customer information business.

While Twilio’s main focus since it launched in 2008 has been on making it easy to embed communications functionality into any app, it signaled a switch in direction when it released the Flex customer service API in March 2018. Later that same year, it bought SendGrid, an email marketing API company for $2 billion.

Twilio’s market cap as of Friday was an impressive $45 billion. You could see how it can afford to flex its financial muscles to combine Twilio’s core API mission, especially Flex, with the ability to pull customer data with Segment and create customized email or ads with SendGrid.

This could enable Twilio to expand beyond pure core communications capabilities and it could come at the cost of around $5 billion for the two companies, a good deal for what could turn out to be a substantial business as more and more companies look for ways to understand and communicate with their customers in more relevant ways across multiple channels.

As Semil Shah from early stage VC firm Haystack wrote in the company blog yesterday, Segment saw a different way to gather customer data, and Twilio was wise to swoop in and buy it.

Segment’s belief was that a traditional CRM wasn’t robust enough for the enterprise to properly manage its pipe. Segment entered to provide customer data infrastructure to offer a more unified experience. Now under the Twilio umbrella, Segment can continue to build key integrations (like they have for Twilio data), which is being used globally inside Fortune 500 companies already.

Segment was founded in 2011 and raised over $283 million, according to Crunchbase data. Its most recent raise was $175 million in April on a $1.5 billion valuation.

Twilio stock closed at $306.24 per share on Friday up $2.39%.

Segment declined to comment on this story. We also sent a request for comment to Twilio, but hadn’t heard back by the time we published.  If that changes, we will update the story.

Sep
25
2019
--

Segment’s new privacy portal helps companies comply with expanding regulations

With the EU’s sweeping GDPR privacy laws and the upcoming California Consumer Privacy ACT (CCPA), companies have to figure out how to deal with keeping private data private — or face massive fines. Segment announced a new Privacy Portal today that could help companies trying to remain in compliance.

Segment CEO and co-founder Peter Reinhardt says companies have built a false dichotomy between personalization and privacy, and he says that it doesn’t have to be that way. “We’ve noticed that a lot of companies feel this tension between privacy and growth. They basically see a paradox between being either privacy-respectful versus providing a very personalized experience,” he said.

The new Privacy Portal is designed to be a central place where customers can sort their data in an automated way and create an inventory of what data they have inside the company. “By introducing a single point of collection for all the data, it creates a choke point on the data collection to allow you to actually govern that, a single place to inspect, monitor, alert and have an inventory of all the data that you’re collecting, so that you can ensure that it’s compliant, and so that you can ensure that you’ve got consent, and all of those things,” he said.

The way this works is that as the data comes into the portal, it automatically gets put into a bucket based on the level of concern about it. “We are basically giving customers monitoring and a consolidated view over all of the different data points that are coming in. So we have matches that basically look for things that might be PII, and we automatically grade most of them with green, yellow or red in terms of the level of potential concern,” Reinhardt explained.

On top of that, companies can apply policies, based on the grades, say letting anything that’s green or yellow through, but preventing any red data (PII) from being shared with other applications.

In addition, to make sure that the product can connect to as many marketing tools as possible to get the most complete data picture, the company is releasing a new feature called Functions, which lets customers build their own custom data connectors. With thousands of marketing technology tools, it’s impossible for Segment to build connectors for all of them. Functions lets companies build custom connectors in a low-code way in instances where Segment doesn’t provide it out of the box.

The two tools are available to Segment customers starting today.

Aug
05
2019
--

Segment CEO Peter Reinhardt is coming to TechCrunch Sessions: Enterprise to discuss customer experience management

There are few topics as hot right now in the enterprise as customer experience management, that ability to collect detailed data about your customers, then deliver customized experiences based on what you have learned about them. To help understand the challenges companies face building this kind of experience, we are bringing Segment CEO Peter Reinhardt to TechCrunch Sessions: Enterprise on September 5 in San Francisco (p.s. early-bird sales end this Friday, August 9).

At the root of customer experience management is data — tons and tons of data. It may come from the customer journey through a website or app, basic information you know about the customer or the customer’s transaction history. It’s hundreds of signals and collecting that data in order to build the experience where Reinhardt’s company comes in.

Segment wants to provide the infrastructure to collect and understand all of that data. Once you have that in place, you can build data models and then develop applications that make use of the data to drive a better experience.

Reinhardt, and a panel that includes Qualtrics’ Julie Larson-Green and Adobe’s Amit Ahuja, will discuss with TechCrunch editors the difficulties companies face collecting all of that data to build a picture of the customer, then using it to deliver more meaningful experiences for them. See the full agenda here.

Segment was born in the proverbial dorm room at MIT when Reinhardt and his co-founders were students there. They have raised more than $280 million since inception. Customers include Atlassian, Bonobos, Instacart, Levis and Intuit .

Early-bird tickets to see Peter and our lineup of enterprise influencers at TC Sessions: Enterprise are on sale for just $249 when you book here; but hurry, prices go up by $100 after this Friday!

Are you an early-stage startup in the enterprise-tech space? Book a demo table for $2,000 and get in front of TechCrunch editors and future customers/investors. Each demo table comes with four tickets to enjoy the show.

Jul
13
2017
--

Marketing data startup Segment raises $64M

Segment co-founders Segment has raised $64 million in Series C funding, with the Y Combinator Continuity Fund and GV leading the round. The continuity fund is the $700 million fund led by former Twitter COO Ali Rowghani. It selectively writes checks for YC startups as they raise bigger rounds. As part of the deal, Rowghani will be joining Segment’s board of directors. In the funding announcement, he said… Read More

Dec
09
2016
--

Segment adds Google BigQuery to expand its customer data platform

Segment BigQuery Segment helps businesses manage all their data from services like Google Analytics, Mixpanel and Salesforce — and send that data to a variety of destinations, whether it’s an attribution product or a data warehouse. The company announced yesterday that it’s expanding its offerings by integrating with BigQuery, Google’s data warehousing service. This might seem like… Read More

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com