Personal blog of Yzmir Ramirez

With its Kubernetes Engine (GKE), Google Cloud has long offered a managed service for running containers on its platform. Kubernetes users tend to have a variety of needs, but so far, Google only offered a single tier of GKE that wasn’t necessarily geared toward the high-end enterprise users the company is trying to woo. Today, however, the company announced a new advanced edition of GKE that introduces a number of new features and an enhanced financially backed SLA, additional security tools and new automation features. You can think of GKE Advanced as the enterprise version of GKE.

The new service will launch in the second quarter of the year and hasn’t yet announced pricing. The regular version of GKE is now called GKE Standard.

Google says the service builds upon the company’s own learnings from running a complex container infrastructure internally for years.

For enterprise customers, the financially backed SLA is surely a nice bonus. The promise here is 99.95 percent guaranteed availability for regional clusters.

Most users who opt for a managed Kubernetes environment do so because they don’t want to deal with the hassle of managing these clusters themselves. With GKE Standard, there’s still some work to be done with regard to scaling the clusters. Because of this, GKE Advanced includes a Vertical Pod Autoscaler that keeps on eye on resource utilization and adjusts it as necessary, as well as Node Auto Provisioning, an enhanced version of cluster autoscaling in GKE Standard.

In addition to these new GKE Advanced features, Google is adding GKE security features like the GKE Sandbox, which is currently in beta and will come exclusively to GKE Advanced once it’s launched, and the ability to enforce that only signed and verified images are used in the container environment.

The Sandbox uses Google’s gVisor container sandbox runtime. With this, every sandbox gets its own user-space kernel, adding an additional layer of security. With Binary Authorization, GKE Advanced users also can ensure that all container images are signed by a trusted authority before they are put into production. Somebody could theoretically still smuggle malicious code into the containers, but this process, which enforces standard container release practices, for example, should ensure that only authorized containers can run in the environment.

GKE Advanced also includes support for GKE usage metering, which allows companies to keep tabs on who is using a GKE cluster and charge them according. This feature, too, will be exclusive to GKE Advanced.

Yesterday, Salesforce .com announced its intent to buy its own educational/nonprofit arm, Salesforce.org, for $300 million. On its face, this feels like a confusing turn of events, but industry experts say it’s really about aligning educational and nonprofit verticals across the entire organization.

Salesforce has always made a lot of hay about being a responsible capitalist. It’s something it highlights at events and really extends with the 1-1-1 model it created, which gives one percent of profit, time and resources (product) to education and nonprofits. Its employees are given time off and are encouraged to work in the community. Salesforce.org has been the driver behind this, but something drove the company to bring Salesforce.org into the fold.

While it’s easy to be cynical about the possible motivations, it could be a simple business reason, says Ray Wang, founder and principal analyst at Constellation Research. As he pointed out, it didn’t make a lot of sense from a business perspective to be running two separate entities with separate executive teams, bookkeeping systems and sales teams. What’s more, he said there was some confusion over lack of alignment and messaging between the Salesforce.com education sales team and what was happening at Salesforce.org. Finally, he says because Salesforce.org couldn’t issue Salesforce.com stock options, it might not have been attracting the best talent.

“It allows them to get better people and talent, and it’s also eliminating redundancies with the education vertical. That was really the big driver behind this,” Wang told TechCrunch.

Tony Byrne, founder and principal analyst at Real Story Group agreed. “My guess is that they were struggling to align roadmaps between the offerings (.com and .org), and they see .org as more strategic now and want to make sure they’re in the fold,” he said.

Focusing on the charity arm

Brent Leary, principal and co-founder at CRM Essentials, says it’s also about keeping that charitable focus front and center, while pulling that revenue into the Salesforce.com revenue stream. “It seems like doing good is set to be really good for business, making it a potentially very good idea to be included as part of Salesforce’s top line revenue numbers, Leary said.

For many, this was simply about keeping up with Microsoft and Google in the nonprofit space, and being part of Salesforce.com makes more sense in terms of competing. “I believe Salesforce’s move to bring Salesforce.org in house was a well-timed strategic move to have greater influence on the company’s endeavors into the Not for Profit (NFP) space. In the wake of Microsoft’s announcements of significantly revamping and adding resources to its Dynamics 365 Nonprofit Accelerator, Salesforce would be well-served to also show greater commitment on their end to helping NFPs acquire greater access to technologies that enable them to carry out their mission,” Daniel Newman, founder and principal analyst at Futurum Research, said.

Good or bad idea?

But not everyone sees this move in a positive light. Patrick Moorhead, principal analyst and founder at Moor Insights and Strategies, says it could end up being a public relations nightmare for Salesforce if the general public doesn’t understand the move. Salesforce could exacerbate that perception if it ends up raising prices for nonprofits and education.

“Salesforce and Benioff’s move with Salesforce.org is a big risk and could blow up in its face. The degree of negative reaction will be dependent on how large the price hikes are and how much earnings get diluted. We won’t know that until more details are released,” Moorhead said.

The deal is still in progress, and will take some months to close, but if it’s simply an administrative move designed to create greater efficiencies, it could make sense. The real question that remains is how this will affect educational and nonprofit institutions as the company combines Salesforce.org and Salesforce.com.

Salesforce did not wish to comment for this story.

Salesforce yesterday announced a move to reposition how it provides software to and works with nonprofits like educational institutions and charities: the company announced that it would integrate Salesforce.org — which had been a reseller of Salesforce software and services to the nonprofit sector — into Salesforce itself as part of a larger, new nonprofit and education vertical. The new vertical, in turn, will be led by Rob Acker, the current CEO of Salesforce.org.

As part of the deal, Salesforce said it would pay $300 million in cash for all shares of Salesforce.org. The latter had existed as a California public benefit corporation, and now it will be converting into a California business corporation.

Salesforce said that the $300 million, in turn, will be distributed to another independent public benefit corporation called the Salesforce.com Foundation, which will use it for philanthropic purposes. Salesforce will be making further contributions to the Foundation, but did not specify the amount.

Salesforce also said that the combination will add between about $150 million and $200 million to the company’s full-year revenues, depending on when the deal closes.

Salesforce.org had been a vehicle for the company to provide nonprofits, educational institutions and philanthropic organizations free or very discounted licenses to use its software, to the tune of some $260 million in grants distributed to over 40,000 organizations. Salesforce will continue that practice, but now that effort, it seems, will come in line with a bigger business operation in which Salesforce will also develop and sell commercial software and services as well.

“Combining Salesforce and Salesforce.org into a new nonprofit and education vertical reinforces the strength of Salesforce’s philanthropic model,” the company notes. “Salesforce will extend this model by continuing to provide free and highly discounted software to nonprofits and education institutions around the world and investing in local communities through employee volunteering, strategic grants and matching employee giving up to $5,000 per employee annually.”

The new organization will include sales, marketing and the company’s Salesforce Customer Success Platform tailored for the nonprofit and education communities, and all future development of the company’s Nonprofit Cloud, Education Cloud and Philanthropy Cloud vertical applications.

Education, nonprofits and philanthropy might not be the most lucrative sectors that come to mind when you think of enterprise IT, but by virtue of their sheer size and ubiquity, and the fact that these organizations also very much need better technology to operate more efficiently, there is a big opportunity.

Some of that will firmly never catapult into the world of big money — and nor should it, in my opinion — but as Newsela and its backer TCV, and Microsoft, identified recently, schools are still big buyers of IT, and the same goes for other nonprofit and philanthropic organizations.

I’m not sure how Salesforce will bring the different sides of the business together, but it makes sense for the company to at least think of them in a more cohesive way, providing financial help where it’s needed and selling where it is not.

Salesforce said that it expects the deal to close in Q2 or Q3 of this year, pending approval from the Attorney General of California and “other customary closing conditions.”

The design world is in a state of full-fledged competition. Never in history have designers and their respective teams had so many options from which to choose. As both demand and supply grow, design players are working to build out the most comprehensive experience possible for their users.

Adobe, the incumbent in the space, is today launching the Adobe Creative Cloud Plugin Accelerator. Essentially, individuals and teams interested in taking some time to build out plugins for Adobe XD can get themselves three months at Adobe’s HQ, access to Adobe’s product, design and engineering team, as well as a $20K per person stipend to offset expenses.

To be clear, Adobe is not taking equity in these projects and participants will leave Adobe HQ with 100 percent ownership over their built IP.

The Adobe Creative Cloud Plugin Accelerator is supported by Adobe’s Fund for Design, a $10 million venture fund launched in May 2018. Both the fund and the accelerator are meant to open up Adobe, which has historically been a more closed ecosystem.

“For a company like Adobe, we’re flexing a new muscle by working with outside parties, in house, at Adobe Headquarters,” said design principal at Adobe Khoi Vinh. “It’s a real change of thinking from the Adobe of five or 10 years ago, but we’re embracing the community’s energy here.”

It was less than a year ago that Adobe opened up Adobe XD to integrate with other tools, such as UserTesting and Airtable, among others.

Vinh says that, for now, Adobe isn’t sure exactly how many teams or individuals it will be accepted into the accelerator. As it’s the first time the company has done something like this, it’s not adhering to a specific number of participants or a rigid curriculum. Vinh says that some teams might have a clear vision of what they’re building and simply seek one-to-one advice from the engineering or product teams, whereas others might want a more collaborative environment to brainstorm and build out the idea itself.

One thing that is clear, however, is that Adobe is looking for hyper-early-stage projects.

“What ended up happening with the Fund for Design is that the grants and investments made a lot of sense for people who were founders and already had companies,” said Vinh. “The Plugin Accelerator is meant to target people who are even earlier-stage than a founder and maybe not ready to start their own company.”

The hope is that teams of one to three will have the chance to build great plug-ins for Adobe XD, making the platform more attractive to clients as Figma and InVision make a run for those same users.

Adobe isn’t the first design tool firm to launch a venture fund. InVision launched the $5 million Design Forward Fund in late 2017.

Folks interested in the Creative Cloud Plugin Accelerator can apply here.

Move over, Flexport. There is another player looking to make waves in the huge and messy business of freight logistics. Zencargo — a London startup that has built a platform that uses machine learning and other new technology to rethink how large shipping companies and their customers manage and move cargo, or freight forwarding as it’s known in the industry — has closed a Series A round of funding of about $19 million.

Zencargo’s co-founder and head of growth Richard Fattal said in an interview that the new funds will be used to continue building its software, specifically to develop more tools for the manufacturers and others who use its platform to predict and manage how cargo is moved around the world.

The Series A brings the total raised by Zencargo to $20 million. This latest round was led by HV Holtzbrinck Ventures . Tom Stafford, managing partner at DST Global; Pentland Ventures; and previous investors Samos, LocalGlobe and Picus Capital also participated in the round.

Zencargo is not disclosing its valuation, nor its current revenues, but Fattal said that in the last 12 months it has seen its growth grow six times over. The company (for now) also does not explicitly name clients, but Fattal notes that they include large e-commerce companies, retailers and manufacturers, including several of the largest businesses in Europe. (One of them at least appears to be Amazon: Zencargo provides integrated services to ship goods to Amazon fulfillment centers.)

Shipping — be it by land, air or sea — is one of the cornerstones of the global economy. While we are increasingly hearing a mantra to “buy local,” the reality of how the mass-market world of trade works is that components for things are not often made in the same place where the ultimate item is assembled, and our on-demand digital culture has created an expectation and competitive market for more than what we can source in our backyards.

For companies like Zencargo, that creates a two-fold opportunity: to ship finished goods — be it clothes, food or anything — to meet those consumer demands wherever they are; and to ship components for those goods — be it electronics, textiles or flour — to produce those goods elsewhere, wherever that business happens to be.

Ironically, while we have seen a lot of technology applied to other aspects of the economics equation — we can browse an app anytime and anywhere to buy something, for example — the logistics of getting the basics to the right place are now only just catching up.

Alex Hersham, another of Zencargo’s co-founders who is also the CEO (the third co-founder is Jan Riethmayer, the CTO), estimates that there is some $1.1 trillion “left on the table” from all of the inefficiencies in the supply chain related to things not being in stock when needed, or overstocked, and other inventory mistakes.

Fattal notes that Zencargo is not only trying to replace things like physical paperwork, faxes and silos of information variously held by shipping companies and the businesses that use them — but the whole understanding and efficiency (or lack thereof) that underlies how everything moves, and in turn the kinds of businesses that can be built as a result.

“Global trade is an enormous market, one of the last to be disrupted by technology,” Fattal said. “We want not just to be a better freight forwarder but we want people to think differently about commerce. Given a choice, where is it best to situate a supplier? Or how much stock do I order? How do I move this cargo from one place to another? When you have a lot of variability in the supply chain, these are difficult tasks to manage, but by unlocking the data in the supply chain you can really change the whole decision making process.”

Zencargo is just getting started on that. Flexport, one of its biggest startup competitors, in February raised $1 billion at a $3.2 billion valuation led by SoftBank to double down on its own freight forwarding business, platform and operations. But as Christian Saller, a partner at HV Holtzbrinck Ventures describes it, there is still a lot of opportunity out there and room for more than one disruptor.

“It’s such a big market that is so broken,” he said. “Right now it’s not about winner-take-all.”

Most work involving computers is highly repetitive, which is why companies regularly have developers write code to automate repetitive tasks. But that process is not very scalable. Ideally, individuals across an entire business would be able to create automated tasks, not just developers. This problem has created a new category called process automation. Startups in this space are all about making companies more efficient.
Most of the existing tools on the market are code-based and complicated, which tends to make it tough for non-technical people to automate anything. Ideally, you would allow them to train software robots to handle repetitive and mundane tasks.

This is the aim of Leapwork, which today announces a Series A investment of $10 million, from London’s DN Capital and e.ventures out of Berlin. The company already has many clients, from tier-one banks and global healthcare firms to aerospace and software companies, and now plans to expand in the U.S. Its customers typically already have a lot of experience with tools such as Tricentis, MicroFocus, UiPath and BluePrism, but employ Leapwork when code-based tools prove limiting.

Founded in 2015 and launched in April 2017, Leapwork has an entirely visual system, backed by a modern tech stack. Instead of using developer time, staff automate tasks themselves, without writing any code, with a simple user interface that is likened to learning PowerPoint or Excel. Leapwork estimates it can save 75 percent of an employee’s time.

Christian Brink Frederiksen, Leapwork’s CEO and co-founder said: “About half of our business comes from the U.S. and this investment will enable us to serve those customers better, as well as reaching new ones.”

Leapwork has found traction in the areas of software testing, data migration and robotic process automation in finance and healthcare. Based in Copenhagen, Denmark, Leapwork has offices in London, U.K., San Francisco, USA, Minsk, Belarus, and Gurugram, India.

Thomas Rubens, of DN Capital, said: “From the outset we were impressed by Leapwork’s product, which we believe will change the automation landscape. Every company has repetitive tasks that could be automated and few have the developer resource to make it happen.”

The founders began in June 2015 in Copenhagen, Denmark, after having worked for almost two decades in enterprise software and business-critical IT. They launched their first pilot in July 2016 and, after working with Global2000 pilot customers in the U.S. and Europe, went live with the Leapwork automation platform in March 2017.

Prior to this funding the company was bootstrapped by the founders, as both had previous successful exits.

Many of our regular attendees already know that the Percona Live Open Source Database Conference is not all about Percona software or, indeed, all about Percona. However, with moving to a new city—Austin, TX— we have realized that it’s something we ought to shout out loud and clear. Our conference really is technology agnostic! As long as submissions were related to open source databases then they were candidate for selection.

We have thirteen tracks at this year’s conference including a track entitled “Other Open Source Databases” which we are presenting alongside tracks dedicated to MySQL®, MariaDB®, MongoDB®, and PostgreSQL. And that’s not all. While most talks are technology-oriented, we also have tracks that are highly relevant if you are managing technology aspects of your business. For those still considering the impact of GDPR you’ll be able to hear talks about other issues relating to compliance and data security that you might well want to get to grips with. Or perhaps consider the talks oriented towards business and enterprise. Maybe you are looking to minimize your license costs by moving from proprietary to open source databases? In which case our migration track might be for you. There are five more tracks for you to discover… why not take a look?

We’d like to thank the volunteer conference committee again for their contributions in developing this fantastic, diverse, and intriguing program!

Companies represented by speakers at Percona Live 2019

Also, of course, not all of the talks are given by Percona speakers. As you can see from this graphic at least sixty companies are represented by speakers at the event, including some huge names not just in the open source space but in the tech space as a whole. Anyone heard of Facebook? Uber? Oracle? Walmart? MailChimp? Alibaba… I won’t list all sixty names, but you get the idea! In fact, both Facebook and Alibaba are sponsoring their own tracks at this year’s conference, alongside PingCap presenting a track dedicated to TiDB. Don’t miss out! Our advanced rate registration ends on Sunday April 21 after which the price moves to standard registration rate. Don’t delay…

Register Now

Sponsors

We base Percona Live events in major cities, use premium venues, and sponsor our own staff to speak…Percona Live is an expensive production and we heavily subsidize the tickets. We are eternally grateful to our sponsors who share the costs of keeping Percona Live special. Without their support it would be very difficult to host an event of this quality and scale.

Diamond sponsors

Platinum sponsors

Gold sponsors

Silver Sponsors

Branding sponsors

Media sponsors

Thanks again to all of our sponsors, we appreciate your support!

Extra Crunch offers members the opportunity to tune into conference calls led and moderated by the TechCrunch writers you read every day. This week, TechCrunch’s Frederic Lardinois and Ron Miller offered up their analysis on the major announcements that came out of Google’s Cloud Next conference this past week, as well as their opinions on the outlook for the company going forward.

Google Cloud announced a series of products, packages and services that it believes will improve the company’s competitive position and differentiate itself from AWS and other peers. Frederic and Ron discuss all of Google’s most promising announcements, including its product for managing hybrid clouds, its new end-to-end AI platform, as well as the company’s heightened effort to improve customer service, communication, and ease-of-use.

“They have all of these AI and machine learning technologies, they have serverless technologies, they have containerization technologies — they have this whole range of technologies.

But it’s very difficult for the average company to take these technologies and know what to do with them, or to have the staff and the expertise to be able to make good use of them. So, the more they do things like this where they package them into products and make them much more accessible to the enterprise at large, the more successful that’s likely going to be because people can see how they can use these.

…Google does have thousands of engineers, and they have very smart people, but not every company does, and that’s the whole idea of the cloud. The cloud is supposed to take this stuff, put it together in such a way that you don’t have to be Google, or you don’t have to be Facebook, you don’t have to be Amazon, and you can take the same technology and put it to use in your company”

Frederic and Ron dive deeper into how the new offerings may impact Google’s market share in the cloud ecosystem and which verticals represent the best opportunity for Google to win. The two also dig into the future of open source in cloud and how they see customer use cases for cloud infrastructure evolving.

For access to the full transcription and the call audio, and for the opportunity to participate in future conference calls, become a member of Extra Crunch. Learn more and try it for free. 

The OpenStack project, which powers more than 75 public and thousands of private clouds, launched the 19th version of its software this week. You’d think that after 19 updates to the open-source infrastructure platform, there really isn’t all that much new the various project teams could add, given that we’re talking about a rather stable code base here. There are actually a few new features in this release, though, as well as all the usual tweaks and feature improvements you’d expect.

While the hype around OpenStack has died down, we’re still talking about a very active open-source project. On average, there were 155 commits per day during the Stein development cycle. As far as development activity goes, that keeps OpenStack on the same level as the Linux kernel and Chromium.

Unsurprisingly, a lot of that development activity focused on Kubernetes and the tools to manage these container clusters. With this release, the team behind the OpenStack Kubernetes installer brought the launch time for a cluster down from about 10 minutes to five, regardless of the number of nodes. To further enhance Kubernetes support, OpenStack Stein also includes updates to Neutron, the project’s networking service, which now makes it easier to create virtual networking ports in bulk as containers are spun up, and Ironic, the bare-metal provisioning service.

All of that is no surprise, given that according to the project’s latest survey, 61 percent of OpenStack deployments now use both Kubernetes and OpenStack in tandem.

The update also includes a number of new networking features that are mostly targeted at the many telecom users. Indeed, over the course of the last few years, telcos have emerged as some of the most active OpenStack users as these companies are looking to modernize their infrastructure as part of their 5G rollouts.

Besides the expected updates, though, there are also a few new and improved projects here that are worth noting.

“The trend from the last couple of releases has been on scale and stability, which is really focused on operations,” OpenStack Foundation executive director Jonathan Bryce told me. “The new projects — and really most of the new projects from the last year — have all been pretty oriented around real-world use cases.”

The first of these is Placement. “As people build a cloud and start to grow it and it becomes more broadly adopted within the organization, a lot of times, there are other requirements that come into play,” Bryce explained. “One of these things that was pretty simplistic at the beginning was how a request for a resource was actually placed on the underlying infrastructure in the data center.” But as users get more sophisticated, they often want to run specific workloads on machines with certain hardware requirements. These days, that’s often a specific GPU for a machine learning workload, for example. With Placement, that’s a bit easier now.

It’s worth noting that OpenStack had some of this functionality before. The team, however, decided to uncouple it from the existing compute service and turn it into a more generic service that could then also be used more easily beyond the compute stack, turning it more into a kind of resource inventory and tracking tool.

Then, there is also Blazer, a reservation service that offers OpenStack users something akin to AWS Reserved Instances. In a private cloud, the use case for a feature is a bit different, though. But as some of the private clouds got bigger, some users found that they needed to be able to guarantee resources to run some of their regular, overnight batch jobs or data analytics workloads, for example.

As far as resource management goes, it’s also worth highlighting Sahara, which now makes it easier to provision Hadoop clusters on OpenStack.

In previous releases, one of the focus areas for the project was to improve the update experience. OpenStack is obviously a very complex system, so bringing it up to the latest version is also a bit of a complex undertaking. These improvements are now paying off. “Nobody even knows we are running Stein right now,” Vexxhost CEO Mohammed Nasar, who made an early bet on OpenStack for his service, told me. “And I think that’s a good thing. You want to be least impactful, especially when you’re in such a core infrastructure level. […] That’s something the projects are starting to become more and more aware of but it’s also part of the OpenStack software in general becoming much more stable.”

As usual, this release launched only a few weeks before the OpenStack Foundation hosts its bi-annual Summit in Denver. Since the OpenStack Foundation has expanded its scope beyond the OpenStack project, though, this event also focuses on a broader range of topics around open-source infrastructure. It’ll be interesting to see how this will change the dynamics at the event.

When you need to upgrade your PostgreSQL databases, there are a number of options available to you. In this post we’ll take a look at how you can upgrade PostgreSQL versions using pg_upgrade, a built-in tool that allows in-place upgrade of your software. Using pg_upgrade allows you, potentially, to minimize your downtime, an essential consideration for many organizations. It also allows you to perform a postgres upgrade with very minimal effort.

In our previous posts, we discussed various methods and tools that can help us perform a PostgreSQL upgrade – (1) pg_dumpall, (2) pg_dump and pg_restore with pg_dumpall, (3) logical replication and pglogical, and (4) slony. Methods 1 and 2 can involve additional downtime compared to the approaches taken in 3 and 4. Whilst performing an upgrade using logical replication or slony may be time consuming and require a lot of monitoring, it can be worth it if you can minimize downtime. If you have large databases that are busy with a lot of transactions, you may be better served using logical replication or slony.

This post is the fifth of our Upgrading or Migrating Your Legacy PostgreSQL to Newer PostgreSQL Versions series. These posts lead up to a live webinar, where we’ll be exploring different methods available to upgrade your PostgreSQL databases. If it’s beyond the live webinar date when you read this, you’ll find the recording at that same link.

pg_upgrade

pg_upgrade (formerly pg_migrator – until PostgreSQL 8.4) is a built-in tool that helps in upgrading a legacy PostgreSQL server to a newer version without the need of a dump and restore. The oldest version from when you can upgrade your PostgreSQL using pg_upgrade is 8.4.x. It is capable of performing faster upgrades by taking into consideration that system tables are the ones that undergo the most change between two major versions. The internal data storage format is less often affected.

In fact, in one of our tests we were able to perform an upgrade of a 2 TB database server from PostgreSQL 9.6.5 to 11.1 in less than 10 seconds. Now that is fast!

Overview of the process

To understand how it works, consider a PostgreSQL server running on 9.3.3 that needs to upgrade to PostgreSQL 11. You should install the latest binaries for your new PostgreSQL version on the server – let’s say PostgreSQL 11.2 – before you begin the upgrade process.

Preparation and consistency checks

Once you have installed the new PostgreSQL version, initialize a new data directory using the new binaries and start it on another port i.e. a different port to the one used by PostgreSQL 9.3.3, in our example. Use pg_upgrade to perform consistency checks between the two servers – PG 9.3.3 and PG 11.2 – running on two different ports. If you get any errors, such as a missing extension, you need to to fix these before you proceeding to the upgrade. Once the consistency check has been passed, you can proceed.

Here is how the log looks if you should get an error while performing consistency checks.

$ /usr/pgsql-11/bin/pg_upgrade -b /usr/pgsql-9.3/bin -B /usr/pgsql-11/bin -d /var/lib/pgsql/9.3/data -D /var/lib/pgsql/11/data_new -c
Performing Consistency Checks on Old Live Server
------------------------------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for invalid "unknown" user columns                 ok
Checking for hash indexes                                   ok
Checking for roles starting with "pg_"                      ok
Checking for incompatible "line" data type                  ok
Checking for presence of required libraries                 fatal
Your installation references loadable libraries that are missing from the
new installation.  You can add these libraries to the new installation,
or remove the functions using them from the old installation.  A list of
problem libraries is in the file:
    loadable_libraries.txt
Failure, exiting
$ cat loadable_libraries.txt
could not load library "$libdir/pg_repack": ERROR:  could not access file "$libdir/pg_repack": No such file or directory

To proceed beyond the error, in this example you’d need to install this missing extension pg_repack for the new PostgreSQL version, and rerun the check to make sure that you receive no errors and all the checks are passed.

Carrying out the upgrade

Once passed, you can proceed in one of two ways. One option is to let pg_upgrade copy the datafiles of the old data directory to the new data directory initialized by the new PostgreSQL version. The second option is to let pg_upgrade use hard links instead of copying data files. Copying a database of several terabytes may be time consuming. Using the hard links method makes the process really quick as it does not involve copying files.

To use hard links with pg_upgrade, you pass an additional argument -k as you can see in the following command.

$ /usr/pgsql-11/bin/pg_upgrade -b /usr/pgsql-9.3/bin -B /usr/pgsql-11/bin -d /var/lib/pgsql/9.3/data -D /var/lib/pgsql/11/data_new -k

In the Unix file system, a file or a directory is a link to an inode (index node) that stores metadata (disk block location, attributes, etc) of the data stored in them. Each inode is identified by an integer or an inode number. When you use pg_upgrade with hard links, it internally creates another file/directory in the new data directory that links to the same inode as it was in the older data directory for that file/directory. So, it skips the physical copy of the objects, but creates each object and links them to the same inode.

This reduces the disk IO and avoids the need for additional space in the server. An important point to note is that this option works only when you are upgrading your PostgreSQL on the same file system. This means, for example, if you want to upgrade to a new or a faster disk during the database upgrade, the hard link option does not work. In that case, you would need to use the file copy method.

So far, we have seen a high level overview of how pg_upgrade with hard links help you to perform an upgrade with lowest possible downtime. Come see more in action during our Webinar.  And don’t forget at Percona Live in Austin, May 28-30 2019, we’ll have two days of PostgreSQL content in a postgres dedicated track.

—
Elephant image based on photo from Pexels