Feb
04
2021
--

Google Cloud launches Apigee X, the next generation of its API management platform

Google today announced the launch of Apigee X, the next major release of the Apgiee API management platform it acquired back in 2016.

“If you look at what’s happening — especially after the pandemic started in March last year — the volume of digital activities has gone up in every kind of industry, all kinds of use cases are coming up. And one of the things we see is the need for a really high-performance, reliable, global digital transformation platform,” Amit Zavery, Google Cloud’s head of platform, told me.

He noted that the number of API calls has gone up 47 percent from last year and that the platform now handles about 2.2 trillion API calls per year.

At the core of the updates are deeper integrations with Google Cloud’s AI, security and networking tools. In practice, this means Apigee users can now deploy their APIs across 24 Google Cloud regions, for example, and use Google’s caching services in more than 100 edge locations.

Image Credits: Google

In addition, Apigee X now integrates with Google’s Cloud Armor firewall and its Cloud Identity Access Management platform. This also means that Apigee users won’t have to use third-party tools for their firewall and identity management needs.

“We do a lot of AI/ML-based anomaly detection and operations management,” Zavery explained. “We can predict any kind of malicious intent or any other things which might happen to those API calls or your traffic by embedding a lot of those insights into our API platform. I think [that] is a big improvement, as well as new features, especially in operations management, security management, vulnerability management and making those a core capability so that as a business, you don’t have to worry about all these things. It comes with the core capabilities and that is really where the front doors of digital front-ends can shine and customers can focus on that.”

The platform now also makes better use of Google’s AI capabilities to help users identify anomalies or predict traffic for peak seasons. The idea here is to help customers automate a lot of the standards automation tasks and, of course, improve security at the same time.

As Zavery stressed, API management is now about more than just managing traffic between applications. But more than just helping customers manage their digital transformation projects, the Apigee team is now thinking about what it calls ‘digital excellence.’ “That’s how we’re thinking of the journey for customers moving from not just ‘hey, I can have a front end,’ but what about all the excellent things you want to do and how we can do that,” Zavery said.

“During these uncertain times, organizations worldwide are doubling-down on their API strategies to operate anywhere, automate processes, and deliver new digital experiences quickly and securely,” said James Fairweather, Chief Innovation Officer at Pitney Bowes. “By powering APIs with new capabilities like reCAPTCHA Enterprise, Cloud Armor (WAF), and Cloud CDN, Apigee X makes it easy for enterprises like us to scale digital initiatives, and deliver innovative experiences to our customers, employees and partners.”

Oct
10
2019
--

Flaw in Cyberoam firewalls exposed corporate networks to hackers

Sophos said it is fixing a vulnerability in its Cyberoam firewall appliances, which a security researcher says can allow an attacker to gain access to a company’s internal network without needing a password.

The vulnerability allows an attacker to remotely gain “root” permissions on a vulnerable device, giving them the highest level of access, by sending malicious commands across the internet. The attack takes advantage of the web-based operating system that sits on top of the Cyberoam firewall.

Once a vulnerable device is accessed, an attacker can jump onto a company’s network, according to the researcher who shared their findings exclusively with TechCrunch.

Cyberoam devices are typically used in large enterprises, sitting on the edge of a network and acting as a gateway to allow employees in while keeping hackers out. These devices filter out bad traffic, and prevent denial-of-service attacks and other network-based attacks. They also include virtual private networking (VPN), allowing remote employees to log on to their company’s network when they are not in the office.

It’s a similar vulnerability to recently disclosed flaws in corporate VPN providers, notably Palo Alto Networks, Pulse Secure and Fortinet, which allowed attackers to gain access to a corporate network without needing a user’s password. Many large tech companies, including Twitter and Uber, were affected by the vulnerable technology, prompting Homeland Security to issue an advisory to warn of the risks.

Sophos, which bought Cyberoam in 2014, issued a short advisory this week, noting that the company rolled out fixes on September 30.

The researcher, who asked to remain anonymous, said an attacker would only need an IP address of a vulnerable device. Getting vulnerable devices was easy, they said, by using search engines like Shodan, which lists around 96,000 devices accessible to the internet. Other search engines put the figure far higher.

A Sophos spokesperson disputed the number of devices affected, but would not provide a clearer figure.

“Sophos issued an automatic hotfix to all supported versions in September, and we know that 99% of devices have already been automatically patched,” said the spokesperson. “There are a small amount of devices that have not as of yet been patched because the customer has turned off auto-update and/or are not internet-facing devices.”

Customers still affected can update their devices manually, the spokesperson said. Sophos said the fix will be included in the next update of its CyberoamOS operating system, but the spokesperson did not say when that software would be released.

The researcher said they expect to release the proof-of-concept code in the coming months.

Aug
08
2019
--

Only 24 hours left to save $100 on TC Sessions: Enterprise 2019

Heads up all you enterprising enterprise software startuppers. You have only 24 hours before the price goes up on tickets to TC Sessions: Enterprise 2019. Save $100 and join us in San Francisco on September 5 — along with some of the industry’s top founders, CEOs, investors and technologists. Buy your early-bird ticket before 11:59 p.m. (PT) on August 9.

Enterprise is, without doubt, Silicon Valley’s 800-pound gorilla. No other startup category is as large, rich or competitive. In this day-long conference, we tackle the big topics and separate hype from reality. Artificial intelligence? Check. Cloud, Kubernetes, security and privacy, marketing automation, quantum? Yes. Investors, founders, and acquisition-hungry big enterprise companies? Tons of opportunity to network efficiently via CrunchMatch? Yeah, all that and more in 20 main-stage sessions — plus separate speaker Q&As and breakout sessions. Check out the day’s agenda.

Here’s a quick example of the type of programming you can expect.

Does the recent Capital One data breach have you up nights worried about the cost and consequences of cyberattacks? Don’t miss TechCrunch editor Zack Whittaker’s interview with Martin Casado (Andreessen Horowitz), Emily Heath (United Airlines) and Wendy Nather (Duo Security) in a session called, Keeping the Enterprise Secure.

Enterprises face a litany of threats from both inside and outside the firewall. Now more than ever, companies — especially startups — have to put security first. From preventing data from leaking to keeping bad actors out of your network, enterprises have it tough. How can you secure the enterprise without slowing growth? We’ll discuss the role of a modern CISO and how to move fast… without breaking things.

Looking for more ways to save or boost your ROI? Look no further. Buy four or more tickets at once and save 20% with the group discount. And, with every ticket you buy to TC Sessions: Enterprise, you’ll score a free Expo Only pass to TechCrunch Disrupt SF on October 2-4.

TC Sessions: Enterprise takes place on September 5, and if you want to save $100, you have just 24 hours left to act. The $249 early-bird ticket price remains in play until 11:59 p.m. (PT) on August 9. Buy your ticket now and save.

Is your company interested in sponsoring or exhibiting at TC Sessions: Enterprise 2019? Contact our sponsorship sales team by filling out this form.

Jul
31
2019
--

Save with group discounts and bring your team to TechCrunch’s first-ever Enterprise event Sept. 5 in SF

Get ready to dive into the fiercely competitive waters of enterprise software. Join more than 1,000 attendees for TC Sessions Enterprise 2019 on September 5 to navigate this rapidly evolving category with the industry’s brightest minds, biggest names and exciting startups.

Our $249 early-bird ticket price remains in play, which saves you $100. But one is the loneliest number, so why not take advantage of our group discount, buy in bulk and bring your whole team? Save an extra 20% when you buy four or more tickets at once.

We’ve packed this day-long conference with an outstanding lineup of presentations, interviews, panel discussions, demos, breakout sessions and, of course, networking. Check out the agenda, which includes both industry titans and boundary-pushing startups eager to disrupt the status quo.

We’ll add more surprises along the way, but these sessions provide a taste of what to expect — and why you’ll need your posse to absorb as much intel as possible.

Talking Developer Tools
Scott Farquhar (Atlassian)

With tools like Jira, Bitbucket and Confluence, few companies influence how developers work as much as Atlassian. The company’s co-founder and co-CEO Scott Farquhar will join us to talk about growing his company, how it is bringing its tools to enterprises and what the future of software development in and for the enterprise will look like.

Keeping the Enterprise Secure
Martin Casado (Andreessen Horowitz), Wendy Nather (Duo Security), Emily Heath (United Airlines)

Enterprises face a litany of threats from both inside and outside the firewall. Now more than ever, companies — especially startups — have to put security first. From preventing data from leaking to keeping bad actors out of your network, enterprises have it tough. How can you secure the enterprise without slowing growth? We’ll discuss the role of a modern CSO and how to move fast — without breaking things.

Keeping an Enterprise Behemoth on Course
Bill McDermott (SAP)

With over $166 billion in market cap, Germany-based SAP is one of the most valuable tech companies in the world today. Bill McDermott took the leadership in 2014, becoming the first American to hold this position. Since then, he has quickly grown the company, in part thanks to a number of $1 billion-plus acquisitions. We’ll talk to him about his approach to these acquisitions, his strategy for growing the company in a quickly changing market and the state of enterprise software in general.

The Quantum Enterprise
Jim Clarke (Intel), Jay Gambetta (IBM
and Krysta Svore (Microsoft)
4:20 PM – 4:45 PM

While we’re still a few years away from having quantum computers that will fulfill the full promise of this technology, many companies are already starting to experiment with what’s available today. We’ll talk about what startups and enterprises should know about quantum computing today to prepare for tomorrow.

TC Sessions Enterprise 2019 takes place on September 5. You can’t be everywhere at once, so bring your team, cover more ground and increase your ROI. Get your group discount tickets and save.

Jul
24
2019
--

Duo’s Wendy Nather to talk security at TC Sessions: Enterprise

When it comes to enterprise security, how do you move fast without breaking things?

Enter Duo’s Wendy Nather, who will join us at TC Sessions: Enterprise in San Francisco on September 5, where we will get the inside track on how to keep enterprise networks secure without slowing growth.

Nather is head of advisory CISOs at Duo Security, a Cisco company, and one of the most respected and trusted voices in the cybersecurity community as a regular speaker on a range of topics, from threat intelligence to risk analysis, incident response, data security and privacy issues.

Prior to her role at Duo, she was the research director at the Retail ISAC, and served as the research director of the Information Security Practice at independent analyst firm 451 Research.

She also led IT security for the EMEA region of the investment banking division of Swiss Bank Corporation — now UBS.

Nather also co-authored “The Cloud Security Rules,” and was listed as one of SC Magazine’s Women in IT Security “Power Players” in 2014.

We’re excited to have Nather discuss some of the challenges startups and enterprises face in security — threats from both inside and outside the firewall. Companies large and small face similar challenges, from keeping data in to keeping hackers out. How do companies navigate the litany of issues and threats without hampering growth?

Who else will we have onstage, you ask? Good question! We’ll be joined by some of the biggest names and the smartest and most prescient people in the industry, including Bill McDermott at SAP, Scott Farquhar at Atlassian, Julie Larson-Green at Qualtrics, Aaron Levie at Box and Andrew Ng at Landing AI and many, many more. See the whole agenda right here.

Early-bird tickets are on sale right now! For just $249 you can see Nather and these other awesome speakers live at TC Sessions: Enterprise. But hurry, early-bird sales end on August 9; after that, prices jump up by $100. Book here.

If you’re a student on a budget, don’t worry, we’ve got a super-reduced ticket for just $75 when you apply for a student ticket right here.

Enterprise-focused startups can bring the whole crew when you book a Startup Demo table for just $2,000. Each table gives you a primo location to be seen by attendees, investors and other sponsors, in addition to four tickets to enjoy the show. We only have a limited amount of demo tables and we will sell out. Book yours here.

Jul
22
2019
--

Announcing the agenda for TC Sessions: Enterprise | San Francisco, September 5

TechCrunch Sessions is back! On September 5, we’re taking on the ferociously competitive field of enterprise software, and thrilled to announce our packed agenda, overflowing with some of the biggest names and most exciting startups in the enterprise industry. And you’re in luck, because $249 early-bird tickets are still on sale — make sure you book yours so you can enjoy all the agenda has to offer.

Throughout the day, you can expect to hear from industry experts and partake in discussions about the potential of new technologies like quantum computing and AI, how to deal with the onslaught of security threats, investing in early-stage startups and plenty more

We’ll be joined by some of the biggest names and the smartest and most prescient people in the industry, including Bill McDermott at SAP, Scott Farquhar at Atlassian, Julie Larson-Green at Qualtrics, Wendy Nather at Duo Security, Aaron Levie at Box and Andrew Ng at Landing AI.

Our agenda showcases some of the powerhouses in the space, but also plenty of smaller teams that are building and debunking fundamental technologies in the industry. We still have a few tricks up our sleeves and will be adding some new names to the agenda over the next month, so keep your eyes open. In the meantime, check out these agenda highlights:

AGENDA

Investing with an Eye to the Future
Jason Green (Emergence Capital), Maha Ibrahim (Canaan Partners) and Rebecca Lynn (Canvas Ventures)
9:35 AM – 10:00 AM

In an ever-changing technological landscape, it’s not easy for VCs to know what’s coming next and how to place their bets. Yet, it’s the job of investors to peer around the corner and find the next big thing, whether that’s in AI, serverless, blockchain, edge computing or other emerging technologies. Our panel will look at the challenges of enterprise investing, what they look for in enterprise startups and how they decide where to put their money.


Talking Shop
Scott Farquhar (Atlassian)
10:00 AM – 10:20 AM

With tools like Jira, Bitbucket and Confluence, few companies influence how developers work as much as Atlassian. The company’s co-founder and co-CEO Scott Farquhar will join us to talk about growing his company, how it is bringing its tools to enterprises and what the future of software development in and for the enterprise will look like.


Q&A with Investors 
10:20 AM – 10:50 AM

Your chance to ask questions of some of the greatest investors in enterprise.


Innovation Break: Deliver Innovation to the Enterprise
DJ Paoni (
SAP), Sanjay Poonen (VMware) and Shruti Tournatory (Sapphire Ventures)
10:20 AM – 10:40 AM

For startups, the appeal of enterprise clients is not surprising — signing even one or two customers can make an entire business, and it can take just a few hundred to build a $1 billion unicorn company. But while corporate counterparts increasingly look to the startup community for partnership opportunities, making the jump to enterprise sales is far more complicated than scaling up the strategy startups already use to sell to SMBs or consumers. Hear from leaders who have experienced successes and pitfalls through the process as they address how startups can adapt their strategy with the needs of the enterprise in mind. Sponsored by SAP.


Coming Soon!
10:40 AM – 11:00 AM


Box’s Enterprise Journey
Aaron Levie (Box)
11:15 AM – 11:35 AM

Box started life as a consumer file-storage company and transformed early on into a successful enterprise SaaS company, focused on content management in the cloud. Levie will talk about what it’s like to travel the entire startup journey — and what the future holds for data platforms.


Bringing the Cloud to the Enterprise
George Brady (Capital One), Byron Deeter (Bessemer Venture Partners) and a speaker to be announced
11:35 AM – 12:00 PM

Cloud computing may now seem like the default, but that’s far from true for most enterprises, which often still have tons of legacy software that runs in their own data centers. What does it mean to be all-in on the cloud, which is what Capital One recently accomplished. We’ll talk about how companies can make the move to the cloud easier, what not to do and how to develop a cloud strategy with an eye to the future.


Keeping the Enterprise Secure
Martin Casado (Andreessen Horowitz), Wendy Nather (Duo Security) and a speaker to be announced
1:00 PM – 1:25 PM

Enterprises face a litany of threats from both inside and outside the firewall. Now more than ever, companies — especially startups — have to put security first. From preventing data from leaking to keeping bad actors out of your network, enterprises have it tough. How can you secure the enterprise without slowing growth? We’ll discuss the role of a modern CSO and how to move fast… without breaking things.


Keeping an Enterprise Behemoth on Course
Bill McDermott (SAP)

1:25 PM – 1:45 PM

With over $166 billion is market cap, Germany-based SAP is one of the most valuable tech companies in the world today. Bill McDermott took the leadership in 2014, becoming the first American to hold this position. Since then, he has quickly grown the company, in part thanks to a number of $1 billion-plus acquisitions. We’ll talk to him about his approach to these acquisitions, his strategy for growing the company in a quickly changing market and the state of enterprise software in general.


How Kubernetes Changed Everything
Brendan Burns (Microsoft), Tim Hockin (Google Cloud), Craig McLuckie (VMware)
and Aparna Sinha (Google)
1:45 PM – 2:15 PM

You can’t go to an enterprise conference and not talk about Kubernetes, the incredibly popular open-source container orchestration project that was incubated at Google. For this panel, we brought together three of the founding members of the Kubernetes team and the current director of product management for the project at Google to talk about the past, present and future of the project and how it has changed how enterprises think about moving to the cloud and developing software.


Innovation Break: Data: Who Owns It
(SAP)

2:15 PM – 2:35 PM

Enterprises have historically competed by being closed entities, keeping a closed architecture and innovating internally. When applying this closed approach to the hottest new commodity, data, it simply does not work anymore. But as enterprises, startups and public institutions open themselves up, how open is too open? Hear from leaders who explore data ownership and the questions that need to be answered before the data floodgates are opened. Sponsored by SAP.


AI Stakes its Place in the Enterprise
Bindu Reddy (Reality Engines), Jocelyn Goldfein (Zetta Venture Partners)
and a speaker to be announced
2:35 PM – 3:00 PM

AI is becoming table stakes for enterprise software as companies increasingly build AI into their tools to help process data faster or make more efficient use of resources. Our panel will talk about the growing role of AI in enterprise for companies big and small.


Q&A with Founders
3:00 PM – 3:30 PM

Your chance to ask questions of some of the greatest startup minds in enterprise technology.


The Trials and Tribulations of Experience Management
Julie Larson-Green (Qualtrics), Peter Reinhardt (Segment) and a speaker to be announced
3:15 PM – 3:40 PM

As companies gather more data about their customers, it should theoretically improve the customer experience, buy myriad challenges face companies as they try to pull together information from a variety of vendors across disparate systems, both in the cloud and on prem. How do you pull together a coherent picture of your customers, while respecting their privacy and overcoming the technical challenges? We’ll ask a team of experts to find out.


Innovation Break: Identifying Overhyped Technology Trends
James Allworth (
Cloudflare), George Mathew (Kespry) and Max Wessel (SAP)
3:40 PM – 4:00 PM

For innovation-focused businesses, deciding which technology trends are worth immediate investment, which trends are worth keeping on the radar and which are simply buzzworthy can be a challenging gray area to navigate and may ultimately make or break the future of a business. Hear from these innovation juggernauts as they provide their divergent perspectives on today’s hottest trends, including Blockchain, 5G, AI, VR and more. Sponsored by SAP.


Fireside Chat
Andrew Ng (Landing AI)
4:00 PM – 4:20 PM

Few technologists have been more central to the development of AI in the enterprise than Andrew Ng . With Landing AI and the backing of many top venture firms, Ng has the foundation to develop and launch the AI companies he thinks will be winners. We will talk about where Ng expects to see AI’s biggest impacts across the enterprise.


The Quantum Enterprise
Jim Clarke (Intel), Jay Gambetta (IBM)
and Krysta Svore (Microsoft)
4:20 PM – 4:45 PM

While we’re still a few years away from having quantum computers that will fulfill the full promise of this technology, many companies are already starting to experiment with what’s available today. We’ll talk about what startups and enterprises should know about quantum computing today to prepare for tomorrow.


Overcoming the Data Glut
Benoit Dageville (Snowflake), Ali Ghodsi (Databricks) and a speaker to be announced
4:45 PM – 5:10 PM

There is certainly no shortage of data in the enterprise these days. The question is how do you process it and put it in shape to understand it and make better decisions? Our panel will discuss the challenges of data management and visualization in a shifting technological landscape where the term “big data” doesn’t begin to do the growing volume justice.


Early-bird tickets are on sale now for just $249. That’s a $100 savings before prices go up — book yours today.

Students, save big with our super discounted $75 ticket when you book here.

Are you a startup? Book a demo table package for just $2,000 (includes 4 tickets) — book here.

Jun
27
2018
--

Webinar 6/28: Securing Database Servers From External Attacks

securing database servers

securing database serversPlease join Percona’s Chief Evangelist Colin Charles on Thursday, June 28th, 2018, as he presents Securing Database Servers From External attacks at 7:00 AM PDT (UTC-7) / 10:00 AM EDT (UTC-4).

 

A critical piece of your infrastructure is the database tier, yet people don’t pay enough attention to it judging by how many are bitten via poorly chosen defaults, or just a lack understanding of running a secure database tier. In this talk, I’ll focus on MySQL/MariaDB, PostgreSQL, and MongoDB, and cover external authentication, auditing, encryption, SSL, firewalls, replication, and more gems from over a decade of consulting in this space from Percona’s 4,000+ customers.

Register Now

 

Colin Charles

Chief Evangelist

Colin Charles is the Chief Evangelist at Percona. He was previously on the founding team of MariaDB Server in 2009, and had worked at MySQL since 2005, and been a MySQL user since 2000. Before joining MySQL, he worked actively on the Fedora and OpenOffice.org projects. He’s well known within open source communities in APAC, and has spoken at many conferences. Experienced technologist, well known in the open source world for work that spans nearly two decades within the community. Pays attention to emerging technologies from an integration standpoint. Prolific speaker at many industry-wide conferences delivering talks and tutorials with ease. Interests: application development, systems administration, database development, migration, Web-based technologies. Considered expert in Linux and Mac OS X usage/administration/roll-out’s. Specialties: MariaDB, MySQL, Linux, Open Source, Community, speaking & writing to technical audiences as well as business stakeholders.

The post Webinar 6/28: Securing Database Servers From External Attacks appeared first on Percona Database Performance Blog.

Jan
15
2018
--

ProxySQL Firewalling

ProxySQL Firewalling

ProxySQL FirewallingIn this blog post, we’ll look at ProxySQL firewalling (how to use ProxySQL as a firewall).

Not long ago we had an internal discussion about security, and how to enforce a stricter set of rules to prevent malicious acts and block other undesired queries. ProxySQL came up as a possible tool that could help us in achieving what we were looking for. Last year I wrote about how to use ProxySQL to stop a single query.

That approach may be good for few queries and as a temporary solution. But what can we do when we really want to use ProxySQL as an SQL-based firewall? And more importantly, how to do it right?

First of all, let us define what “right” can be in this context.

For right I mean an approach that allows us to have rules matching as specifically as possible, and impacting the production system as little as possible.

To make this clearer, let us assume I have three schemas:

  • Shakila
  • World
  • Windmills

I want to have my firewall block/allow SQL access independently by each schema, user, eventually by source, and so on.

There are a few case where this is not realistic, like in SaaS setups where each schema represents a customer. In this case, the application will have exactly the same kind of SQL – just pointing to different schemas depending the customer.

Using ProxySQL

Anyhow… ProxySQL allows you to manage query firewalling in a very simple and efficient way using the query rules.

In the mysql_query_rules table, we can define a lot of important things – one being setting our SQL firewall.

How?

Let us take a look to the mysql_query_rules table:

rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
    username VARCHAR,
    schemaname VARCHAR,
    flagIN INT NOT NULL DEFAULT 0,
    client_addr VARCHAR,
    proxy_addr VARCHAR,
    proxy_port INT,
    digest VARCHAR,
    match_digest VARCHAR,
    match_pattern VARCHAR,
    negate_match_pattern INT CHECK (negate_match_pattern IN (0,1)) NOT NULL DEFAULT 0,
    re_modifiers VARCHAR DEFAULT 'CASELESS',
    flagOUT INT,
    replace_pattern VARCHAR,
    destination_hostgroup INT DEFAULT NULL,
    cache_ttl INT CHECK(cache_ttl > 0),
    reconnect INT CHECK (reconnect IN (0,1)) DEFAULT NULL,
    timeout INT UNSIGNED,
    retries INT CHECK (retries>=0 AND retries <=1000),
    delay INT UNSIGNED,
    next_query_flagIN INT UNSIGNED,
    mirror_flagOUT INT UNSIGNED,
    mirror_hostgroup INT UNSIGNED,
    error_msg VARCHAR,
    OK_msg VARCHAR,
    sticky_conn INT CHECK (sticky_conn IN (0,1)),
    multiplex INT CHECK (multiplex IN (0,1,2)),
    log INT CHECK (log IN (0,1)),
    apply INT CHECK(apply IN (0,1)) NOT NULL DEFAULT 0,
    comment VARCHAR)

We can define rules around almost everything: connection source, port, destination IP/Port, user, schema, SQL text or any combination of them.

Given we may have quite a large set of queries to manage, I prefer to logically create “areas” around which add the rules to manage SQL access.

For instance, I may decide to allow a specific set of SELECTs to my schema windmills, but nothing more.

Given that, I allocate the set of rule IDs from 100 to 1100 to my schema, and add my rules in three groups.

  1. The exception that will bypass the firewall
  2. The blocking rule(s) (the firewall)
  3. The managing rules (post-processing, like sharding and so on)

There is a simple thing to keep in mind when you design rules for firewalling: do you need post-processing of the query or not?

In the case that you DON’T need post-processing, the rule can simply apply and exit the QueryProcessor. That is probably the most common scenario, and read/write splits can be defined in the exception rules assigned to the rule for the desired HostGroup.

If you DO need post-processing, the rule MUST have apply=0 and the FLAGOUT must be defined. That allows you to have additional actions once the query is beyond the firewall. An example is in case of sharding, where you need to process the sharding key/comment or whatever.

I will use the simple firewall scenario, given this is the topic of the current article.

The rules

Let us start with the easy one, set 2, the blocking rule:

insert into mysql_query_rules (rule_id,username,schemaname,match_digest,error_msg,active,apply) values(1000,'pxc_test','windmills','.','You cannot pass.....I am a servant of the Secret Fire, wielder of the flame of Anor,. You cannot pass.',1, 1);

In this query rule, I had defined the following:

  • User connecting
  • Schema name
  • Any query
  • Message to report
  • Rule_id

That rule will block ANY query that tries to access the schema windmills from application user pxc_test.

Now in set 1, I will add all the rules I want to let pass. I will report here one only, but all can be found in GitHub here (https://github.com/Tusamarco/blogs/tree/master/proxysql_firewall).

insert into mysql_query_rules (rule_id,proxy_port,username,destination_hostgroup,schemaname,active,retries,apply,flagout,match_digest) values(101,6033,'pxc_test',52,'windmills',1,3,1,1000,'SELECT wmillAUTOINC.id,wmillAUTOINC.millid,wmillAUTOINC.location FROM wmillAUTOINC WHERE wmillAUTOINC.millid=.* and wmillAUTOINC.active=.*');

That is quite simple and straightforward, but there is an important element that you must note. In this rule, apply must have value of =1 always, to allow the query rule to bypass without further delay the firewall.

(Side Note: if you need post-processing, the flagout needs to have a value (like flagout=1000) and apply must be =0. That allows the query to jump to set 3, the managing rules.)

This is it, ProxySQL will go to the managing rules as soon as it finds a matching rule that allows the application to access my database/schema, or it will exit if apply=1.

A graph will help to understand better:

Rule set 3 has the standard query rules to manage what to do with the incoming connection, like sharding or redirecting SELECT FOR UPDATE, and so on:

insert into mysql_query_rules (rule_id,proxy_port,schemaname,username,destination_hostgroup,active,retries,match_digest,apply,flagin) values(1040,6033,'windmills','pxc_test',50,1,3,'^SELECT.*FOR UPDATE',1,1000);

Please note the presence of the flagin, which matches the flagout above.

Setting rules, sometimes thousands of them, can be very confusing. It is very important to correctly plan what should be in as an excluding rule and what should not. Do not rush, take your time and identify the queries you need to manage carefully.

Once more ProxySQL can help us. Querying the table stats_mysql_query_digest tells us exactly what queries were sent to ProxySQL:

admin@127.0.0.1) [main]>select hostgroup,schemaname,digest,digest_text,count_star from stats_mysql_query_digest where schemaname='windmills' order by count_star desc;

The above query shows us all the queries hitting the windmills schema. From there we can decide which queries we want to pass and which not.

>select hostgroup,schemaname,digest,digest_text,count_star from stats_mysql_query_digest where schemaname='windmills' order by count_star desc  limit 1G
*************************** 1. row ***************************
  hostgroup: 50
 schemaname: windmills
     digest: 0x18CA8FF2C9C53276
digest_text: SHOW GLOBAL STATUS
 count_star: 141

Once we have our set done (check on github for an example), we are ready to check how our firewall works.

By default, I suggest you to keep all the exceptions (in set 1) with active=0, just to test the firewall.

For instance, my application generates the following exception:

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You cannot pass.....I am a servant of the Secret Fire, wielder of the flame of Anor,. You cannot pass.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
	at com.mysql.jdbc.Util.getInstance(Util.java:386)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4187)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4119)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2570)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2731)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2809)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2758)
	at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1612)
	at net.tc.stresstool.statistics.providers.MySQLStatus.getStatus(MySQLStatus.java:48)
	at net.tc.stresstool.statistics.providers.MySQLSuper.collectStatistics(MySQLSuper.java:92)
	at net.tc.stresstool.statistics.StatCollector.collectStatistics(StatCollector.java:258)
	at net.tc.stresstool.StressTool.<init>(StressTool.java:198)
	at net.tc.stresstool.StressTool.main(StressTool.java:282)

Activating the rules, will instead allow your application to work as usual.

What is the impact?

First, let’s define the baseline by running the application without any rule blocking (but only the r/w split (set 3)).

Queries/sec:

Queries/sec global

Using two application servers:

  • Server A: Total Execution time = 213
  • Server B: Total Execution time = 209

Queries/sec per server

As we can see, queries are almost equally distributed.

QueryProcessor time taken/Query processed total

All queries are processed by QueryProcessor in ~148ms AVG (total).

QueryProcessor efficiency per query

The single query cost is in nanoseconds (avg 10 us).

Use match_digest

Once we’ve defined the baseline, we can go ahead and activate all the rules using the match_digest. Run the same tests again and… :

Using two application servers:

  • Server A: Total Execution time = 207
  • Server B: Total Execution time = 204

First of all, we notice that the execution time did not increase. This is mainly because we have CPU cycles to use in the ProxySQL box:

Here we have a bit of unbalance. We will investigate that in a separate session, but all in all, time/effort looks ok:

Here we have the first thing to notice. Comparing this to the baseline we defined, we can see that using the rules as match_digest significantly increased the execution time to 458ms:

Also notice that if we are in the range of nanoseconds, the cost of processing the query is now three times that of the baseline. Not too much, but if you add a stone to another stone and another stone and another stone … you end up building a huge wall.

So, what to do? Up to now, we saw that managing the firewall with ProxySQL is easy and it can be set at very detailed level – but the cost may not be what we expect it to be.

What can be done? Use DIGEST instead

The secret is to not use match_digest (which implies interpretation of the string) but to use the DIGEST of the query (which is calculated ahead and remains constant for that query).

Let us see what happens if we run the same load using DIGEST in the MYSQL_QUERY_RULES table:

Using two application servers:

  • Server A: Total Execution time = 213
  • Server B: Total Execution time = 209

No, this is not an issue with cut and paste. I had more or less the same execution time as without rules, at the seconds (different millisecond though):

Again, there is some unbalance, but a minor thing:

And we drop to 61ms for execution of all queries. Note that we improve the efficiency of the Query Processor from 148ms AVG to 61ms AVG.

Why? Because our rules using the DIGEST also have the instructions for read/write split, so requests can exit the Query Processor with all the information required at this stage (much more efficient).

Finally, when using the DIGEST the cost for query drops to 4us which is … LOW!

That’s it! ProxySQL using the DIGEST field from mysql_query_rules performs much better given that it doesn’t need to analyze the whole SQL string with regular expressions – it just matches the DIGEST.

Conclusions

ProxySQL can be effectively used as an SQL firewall, but some best practices should be taken in to consideration. First of all, try to use specific rules and be specific on what should be filtered/allowed. Use filter by schema or user or IP/port or combination of them. Always try to avoid match_digest and use digest instead. That allows ProxySQL to bypass the call to the regularExp lib and is far more efficient. Use stats_mysql_query_digest to identify the correct DIGEST.

Regarding this, it would be nice to have a GUI interface that allows us to manage these rules. That would make the usage of the ProxySQL much easier, and the maintenance/creation of rule_chains friendlier.

Feb
25
2014
--

Ex-Googler-Founded Shape Security Picks Up Another $40M To Build Out Its “Botwall” For Businesses

Shape Security — the enterprise startup that emerged from stealth last month with an enterprise product that fights automated malware and bots by way of a firewall (or ‘botwall’ in its words) that shifts its shape depending on what is trying to scale it — has picked up another $40 million in funding. Read More

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com