Personal blog of Yzmir Ramirez

Microsoft today announced the preview launch of Azure Dedicated Host, a new cloud service that will allow you to run your virtual machines on single-tenant physical services. That means you’re not sharing any resources on that server with anybody else and you’ll get full control over everything that’s running on that machine.

Previously, Azure already offered isolated Virtual Machine sizes for two very large virtual machine types. Those are still available, but their use cases are comparably limited to these new hosts, which offer far more flexibility.

With this move, Microsoft is following in the footsteps of AWS, which also offers Dedicated Hosts with very similar capabilities. Google Cloud, too, offers what it calls “sole-tenant nodes.”

Azure Dedicated Host will support Windows, Linux and SQL Server virtual machines and pricing is per host, independent of the number of virtual machines you end up running on them. You can currently opt for machines with up to 48 physical cores and prices start at $4.039 per hour.

To do this, Microsoft is offering two different processors to power these machines. Type 1 is based on the 2.3 GHz Intel Xeon E5-2673 v4 with up to 3.5 gigahertz of clock speed, while Type 2 features the Intel Xeon® Platinum 8168 with single-core clock speeds of up to 3.7 gigahertz. The available memory ranges from 144GiB to 448GiB. You can find more details here.

As Microsoft notes, these new dedicated hosts can help companies reach their compliance requirements for physical security, data integrity and monitoring. The dedicated hosts still share the same underlying infrastructure as any other host in the Azure data centers, but users have full control over any maintenance window that could impact their servers.

These dedicated hosts can also be grouped into larger host groups in a given Azure region, allowing you to build clusters of your own physical servers inside the Azure data center. Because you’re actually renting a physical machine, any hardware issue on that machine will impact the virtual machines you are running on them, so chances are you’ll want to have multiple dedicated hosts for your failover strategy anyway.

In this post, we’ll address how MaxScale monitors servers. We saw in the

We saw in the previous post how we could deal with high availability (HA) and read-write split using MaxScale.

If you remember from the previous post, we used this section to monitor replication:

[Replication Monitor]
type=monitor
module=mysqlmon
servers=percona1, percona2, percona3
user=maxscale
passwd=264D375EC77998F13F4D0EC739AABAD4
monitor_interval=1000
script=/usr/local/bin/failover.sh
events=master_down

But what are we monitoring? We are monitoring the assignment of master and slave roles inside MaxScale according to the actual replication tree in the cluster using the default check from the

mysqlmon

 monitoring modules.

There are other monitoring modules available with MaxScale:

• galera monitor
• multi-master monitor
• ndb cluster monitor

So back to our setup. MaxScale monitors the roles of our servers involved in replication. We can see the status of every server like this:

# maxadmin -pmariadb show server percona2
Server 0x1cace90 (percona2)
	Server:                              192.168.90.3
	Status:                              Slave, Running
	Protocol:                    MySQLBackend
	Port:                                3306
	Server Version:			5.6.28-76.1-log
	Node Id:                     2
	Master Id:                   1
	Slave Ids:
	Repl Depth:                  1
	Number of connections:               0
	Current no. of conns:                0
	Current no. of operations:   0

Now if we stop the slave, we can see:

# maxadmin -pmariadb show server percona2
Server 0x1cace90 (percona2)
	Server:                              192.168.90.3
	Status:                              Running
	Protocol:                    MySQLBackend
	Port:                                3306
	Server Version:			5.6.28-76.1-log
	Node Id:                     2
	Master Id:                   -1
	Slave Ids:
	Repl Depth:                  -1
	Number of connections:               40
	Current no. of conns:                0
	Current no. of operations:   0
# maxadmin -pmariadb list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server             | Address         | Port  | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
percona1           | 192.168.90.2    |  3306 |           0 | Master, Running
percona2           | 192.168.90.3    |  3306 |           0 | Running
percona3           | 192.168.90.4    |  3306 |           0 | Slave, Running
-------------------+-----------------+-------+-------------+--------------------

and in the MaxScale logs:

2016-02-23 14:29:09   notice : Server changed state: percona2[192.168.90.3:3306]: lost_slave

Now if the slave is lagging, nothing happens, and we will then keep sending reads to a slave that is not up to date

To avoid that situation, we can add to the “[Replication Monitor]” section the following parameter:

detect_replication_lag=true

If we do so, MaxScale (if it has enough privileges) will create a schema

maxscale_schema

  with a table

replication_heartbeat

 . This table will be used to verify the replication lag like pt-heartbeat does.

When enabled, after we restart MaxScale, we can see the slave lag:

# maxadmin -pmariadb show server percona2
Server 0x2784f00 (percona2)
	Server:                              192.168.90.3
	Status:                              Slave, Running
	Protocol:                    MySQLBackend
	Port:                                3306
	Server Version:			5.6.28-76.1-log
	Node Id:                     2
	Master Id:                   1
	Slave Ids:
	Repl Depth:                  1
	Slave delay:		670
	Last Repl Heartbeat:	Tue Feb 23 14:25:24 2016
	Number of connections:               0
	Current no. of conns:                0
	Current no. of operations:   0

Does this mean that now the node won’t be reached (no queries will be routed to it)?

Let’s check:

percona3 mysql> select @@hostname;
+------------+
| @@hostname |
+------------+
| percona2   |
+------------+

That doesn’t sound good…

# maxadmin -pmariadb show server percona2
Server 0x2784f00 (percona2)
	Server:                              192.168.90.3
	Status:                              Slave, Running
	Protocol:                    MySQLBackend
	Port:                                3306
	Server Version:			5.6.28-76.1-log
	Node Id:                     2
	Master Id:                   1
	Slave Ids:
	Repl Depth:                  1
	Slave delay:		1099
	Last Repl Heartbeat:	Tue Feb 23 14:25:24 2016
	Number of connections:               1
	Current no. of conns:                1
	Current no. of operations:   0

We can see that there is

1 current connection

 .

How come? The monitoring actually works as expected, but we didn’t configure our

Splitter Service

  to not use that lagging slave.

We need to configure it like this:

[Splitter Service]
type=service
router=readwritesplit
servers=percona1, percona2
max_slave_replication_lag=30
...

And now, if the slave lags for 30 seconds or more, it won’t be used.

But what happen if for any reason we need to stop all the slaves (or if replication breaks)?

To find out, I performed a

STOP SLAVE;

  on percona2 and percona3. This what we see in the logs:

2016-02-23 22:55:16   notice : Server changed state: percona2[192.168.90.3:3306]: lost_slave
2016-02-23 22:55:34   notice : Server changed state: percona1[192.168.90.2:3306]: lost_master
2016-02-23 22:55:34   notice : Server changed state: percona3[192.168.90.4:3306]: lost_slave
2016-02-23 22:55:34   error  : No Master can be determined. Last known was 192.168.90.2:3306
2016-02-23 22:55:45   error  : Couldn't find suitable Master from 2 candidates.
2016-02-23 22:55:45   error  : 140003532506880 [session_alloc] Error : Failed to create Splitter Service session because routercould not establish a new router session, see earlier error.
2016-02-23 22:55:46   error  : Couldn't find suitable Master from 2 candidates.
2016-02-23 22:55:46   error  : 140003542996736 [session_alloc] Error : Failed to create Splitter Service session because routercould not establish a new router session, see earlier error.

If there are no more slaves, the master is not a master anymore, and the routing doesn’t work. The service is unavailable!

As soon as we start a slave, the service is back:

2016-02-23 22:59:17   notice : Server changed state: percona3[192.168.90.4:3306]: new_slave
2016-02-23 22:59:17   notice : A Master Server is now available: 192.168.90.2:3306

Can we avoid this situation when all slaves are stopped?

Yes we can, but we need to add into the monitoring section the following line:

detect_stale_master=true

If we stop  the two slaves again, in MaxScale’s log we can now read:

2016-02-23 23:02:19   notice : Server changed state: percona2[192.168.90.3:3306]: lost_slave
2016-02-23 23:02:46   warning: [mysql_mon]: root server [192.168.90.2:3306] is no longer Master, let's use it again even  if it could be a stale master, you have been warned!
2016-02-23 23:02:46   notice : Server changed state: percona3[192.168.90.4:3306]: lost_slave

And we can still connect to our service and use the single master.

Next time we will see how the read-write split works.

The CVE-2015-0204 FREAK SSL vulnerability abuses intentionally weak “EXPORT” ciphers which could be used to perform a transparent Man In The Middle attack. (We seem to be continually bombarded with not only SSL vulnerabilities but the need to name vulnerabilities with increasing odd names.)

Is your server vulnerable?

This can be tested using the following GIST

If the result is 0; the server is not providing the EXPORT cipher; and as such is not vulnerable.

Is your client vulnerable?

Point your client to https://oneiroi.co.uk:4443/test if this returns “Vulnerable” then the client is vulnerable, if you find a connection error your client should not be vulnerable for example:

root@host:/tmp$ openssl version
OpenSSL 1.0.1e 11 Feb 2013
root@host:/tmp$ curl https://oneiroi.co.uk:4443/test -k
Vulnerable

root@host:/tmp$ openssl s_client -connect oneiroi.co.uk:4443
CONNECTED(00000003)
depth=0 C = XX, L = Default City, O = Default Company Ltd
verify error:num=18:self signed certificate
verify return:1
depth=0 C = XX, L = Default City, O = Default Company Ltd
verify return:1
—
Certificate chain
0 s:/C=XX/L=Default City/O=Default Company Ltd
i:/C=XX/L=Default City/O=Default Company Ltd
—
Server certificate
—–BEGIN CERTIFICATE—–
MIIDVzCCAj+gAwIBAgIJANvTn7jl
…

[root@3654e4df1cc2 bin]# curl https://oneiroi.co.uk:4443/test -k
curl: (35) Cannot communicate securely with peer: no common encryption algorithm(s).
[root@3654e4df1cc2 bin]# openssl s_client -connect oneiroi.co.uk:4443
CONNECTED(00000003)
139942442694560:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:744:
…
In short a vulnerable client will complete the connection, and a non vulnerable client should present an SSL handshake failure error.

DIY

You can recreate this setup yourself


openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout mycert.pem -out mycert.pem;
openssl s_server -cipher EXPORT -accept 4443 -cert mycert.pem -HTTP;


Is MySQL affected ?

Some of the code per the POODLE Blog post can be re-purposed here.


mysql -Bse "SHOW STATUS LIKE 'Ssl_cipher_list'" | sed 's/:/n/g' | grep EXP | wc -l


A result of 0 means the MySQL instance does not support any of the EXPORT ciphers, and thus should not be vulnerable to this attack.

How about other clients?

Most clients link to another library for SSL purposes; however there are examples where this is not the case; take for example golang http://golang.org/pkg/crypto/tls/ which partially implements the TLS1.2 RFC.

The following test code however shows golang does not appear to be affected.


package main

import (
“fmt”
“net/http”
“crypto/tls”
)

func main() {
tr := &http.Transport{
TLSClientConfig: &tls.Config{},
DisableCompression: true,
}
client := &http.Client{Transport: tr}
resp, err := client.Get(“https://oneiroi.co.uk:4443/test”)
fmt.Println(err)
fmt.Println(resp)
}

Get https://oneiroi.co.uk:4443/test: remote error: handshake failure

SSLLabs

Qualys’s SSLLabs now have a test avaialble here: https://dev.ssllabs.com/ssltest/viewMyClient.html

References

• https://www.smacktls.com/
• https://freakattack.com/
• http://blog.cryptographyengineering.com/2015/03/attack-of-week-freak-or-factoring-nsa.html
• https://blogs.akamai.com/2015/03/cve-2015-0204-getting-out-of-the-export-business.html

The post How to test if CVE-2015-0204 FREAK SSL security flaw affects you appeared first on MySQL Performance Blog.