Planet MariaDB

January 19, 2018

MariaDB Foundation

MariaDB 5.5.59 now available

The MariaDB project is pleased to announce the immediate availability of MariaDB 5.5.59. This is a stable (GA) release. See the release notes and changelog for details. Download MariaDB 5.5.59 Release Notes Changelog What is MariaDB 5.5? MariaDB APT and YUM Repository Configuration Generator Thanks, and enjoy MariaDB!

The post MariaDB 5.5.59 now available appeared first on MariaDB.org.

by Ian Gilfillan at January 19, 2018 04:18 PM

MariaDB AB

MariaDB Server 5.5.59 now available

MariaDB Server 5.5.59 now available dbart Fri, 01/19/2018 - 11:07

The MariaDB project is pleased to announce the immediate availability of MariaDB Server 5.5.59. See the release notes and changelog for details and visit mariadb.com/downloads to download.

Download MariaDB Server 5.5.59

Release Notes Changelog What is MariaDB 5.5?

The MariaDB project is pleased to announce the immediate availability of MariaDB Server 5.5.59. See the release notes and changelog for details.

Login or Register to post comments

by dbart at January 19, 2018 04:07 PM

Oli Sennhauser

Short term notice: Percona XtraDB Cluster training in English 7/8 February 2018 in Germany

FromDual offers short term a Percona XtraDB Cluster and MySQL Galera Cluster training (2 days) in English.

The training will take place in the Linuxhotel in Essen/Germany on February 7/8 2018.

There are already enough registrations so it is certain the training will take place. But there are still free places for some additional participants.

You can book online at the Linuxhotel.

Important: The Linuxhotel is nearly fully booked out. So accommodation is in nearby locations. The Linuxhotel will recommend you some locations.

The training is in English.

You can find the contents of this 2-day Percona XtraDB Cluster training here.

If you have any question please do not hesitate to contact us.

by Shinguz at January 19, 2018 04:05 PM

January 18, 2018

MariaDB Foundation

MariaDB 10.3.4 and MariaDB Connector/C 3.0.3 and 2.3.5 now available

The MariaDB project is pleased to announce the availability of MariaDB 10.3.4, the second beta release in the MariaDB 10.3 series, as well as MariaDB Connector/C 3.0.3, the latest stable release in the MariaDB Connector/J 3.0 series, and MariaDB Connector/C 2.3.5, the latest stable release in the MariaDB Connector/C 2.3 series. See the release notes […]

The post MariaDB 10.3.4 and MariaDB Connector/C 3.0.3 and 2.3.5 now available appeared first on MariaDB.org.

by Ian Gilfillan at January 18, 2018 05:00 PM

Jean-Jerome Schmidt

New Video - Ten Tips to Secure MySQL & MariaDB

This video, based on last weeks blog “Ten Tips to Achieve MySQL and MariaDB Security”, walks you through ten different items to keep in mind when deploying a MySQL or MariaDB database to production.

Database security is an essential part of any system. With more and more news reports of widespread data breaches coming in from around the world, there is no better time to check your environments and make sure you have implemented these basic steps to remain secure.

ClusterControl for Database Security

ClusterControl provides advanced deployment, monitoring and management features to ensure your databases and their data are secure. It ensures that your open source database deployments always adhere to basic security model setups for each technology.

ClusterControl provides the Package Summary Operational Report that shows you how many technology and security patches are available to upgrade and can even execute the upgrades for you!

In addition ClusterControl offers…

  • Secure Deployments
    Every technology has its own unique security features and ClusterControl ensures that what should be enabled is enabled during deployment. This eliminates the risk of human error which could otherwise result in leaving the database vulnerable because of a security setting oversight.
  • Communication Security
    ClusterControl provides the ability to install a purchased or self-signed SSL certificate to encrypt the communications between the server and the client. Replication traffic within a Galera Cluster can also be encrypted. Keys for these certificates are entered into and managed by ClusterControl.
  • Backup Security
    Backups are encrypted at rest using AES-256 CBC algorithm. An auto generated key will be stored in the cluster's configuration file under /etc/cmon.d. The backup files are transferred in encrypted format. Users can now secure their backups for offsite or cloud storage with the flip of a checkbox. This feature is available for select backup methods for MySQL, MongoDB & PostgreSQL.
  • User Management
    ClusterControl’s advanced user management features allow you to restrict read or write access to your data at the database or table level. ClusterControl also provides advisors that check that all of your users have proper passwords, and even comes with checks to make sure any part of your database is not open to the public.
  • Reports & Auditing
    ClusterControl provides reporting and audit tools to ensure you remain compliant, whether it is to an industry standard or to your own requirements. It also provides several Developer Studio Advisors that check your database environment to ensure that it is secure. You can even create your own security advisors to automate your own best practices. In addition, several Operational Reports found in ClusterControl can provide you with information you need to know to ensure your database environment is secure.

Download ClusterControl today to take advantage of these database security features.

by fwlymburner at January 18, 2018 12:27 PM

MariaDB AB

Now with System Versioned Tables - Announcing MariaDB Server 10.3 Second Beta

Now with System Versioned Tables - Announcing MariaDB Server 10.3 Second Beta RalfGebhardt Thu, 01/18/2018 - 07:19

We are happy to announce the second beta release of MariaDB Server 10.3, the fastest growing open source relational database. Beta is an important time in our release and we encourage you to download this release today! Please note that we do not recommend running beta releases in production.

MariaDB Server 10.2 added enhancements like Window Functions, Common Table Expressions, JSON functions and CHECK constraints. MariaDB Server 10.3 is the next evolution. For MariaDB Server 10.3 a lot of effort has been spent on database compatibility enhancements, especially for stored routines. This will allow easier migration of stored functions and better usability of stored functions in general.

With System Versioned Tables we are now adding a new powerful enhancement to MariaDB Server, the ability to process temporal data. This opens new use cases to MariaDB Server, like retrospective and trend data analysis, forensic discovery, or data auditing.
System Versioned Tables could  be used for compliance, audit, risk analysis, or position analysis. 

Enabling the System Versioned Tables feature is as easy as altering an existing table:

ALTER TABLE products ADD SYSTEM VERSIONING;

or when creating a new table:

CREATE TABLE products (
pname VARCHAR(30), price decimal(8,2)
) WITH SYSTEM VERSIONING;

System versioned tables are storing timestamps for when data has been added until it has been updated or deleted. This allows to query the data "as of" a given time, or to compare the data "as of" a different date and time.

SELECT * FROM products FOR SYSTEM_TIME AS OF TIMESTAMP @t1;

Now, with MariaDB Server 10.3.4 beta, several significant features and enhancements are available for our users and customers, including:

  • Temporal Data Processing
  • Database Compatibility Enhancements
    • PL/SQL Compatibility for MariaDB Stored Functions: The server now understands a subset of Oracle's PL/SQL language instead of the traditional MariaDB syntax for stored routines
    • New option for CURSOR in stored routines: A CURSOR can now have parameters used by the associated query
    • New data types for stored routines: ROW data type, TYPE OF and ROW TYPE OF anchored data types
    • Generation of unique primary keys by SEQUENCES: As an alternative to AUTO INCREMENT It is now possible to define names sequence objects to create a sequence of numeric values
    • Operations over result sets with INTERSECT and EXCEPT: In addition to the already existing UNION an intersection and subtraction of result sets is now possible
    • Define Columns to be invisible: Columns now can be defined to be invisible. There exist 3 levels of invisibility, user defined, system level and completely invisible
    • Window Function Enhancement: percentile and median window functions have been added
  • User Flexibility
    • User Defined Aggregate Functions: In addition to creating SQL functions it is now also possible to create aggregate functions
    • Lifted limitations for updates and deletes: A DELETE statement can now delete from a table used in the WHERE clause. UPDATE can be the same for source and target
  • Performance/Storage Enhancements
  • Storage Engine Enhancements
    • Spider Storage Engine: The partitioning storage engine has been updated to the newest release of the Spider Storage engine to support new Spider features including direct join support, direct update and delete, direct aggregates
    • Proxy Layer Support for MariaDB Server: Client / Server authentication via a Proxy like MariaDB MaxScale using a Server Proxy Protocol Support

Try out MariaDB Server Beta software and share your feedback!

Download MariaDB Server 10.3.4 Beta

Release Notes Changelog What is MariaDB Server 10.3?

 

We are happy to announce the second beta release of MariaDB Server 10.3, the fastest growing open source relational database.

With System Versioned Tables we are now adding a new powerful enhancement to MariaDB Server, the ability to process temporal data. This opens new use cases to MariaDB Server, like retrospective and trends data analysis, forensic discovery or data auditing.

Login or Register to post comments

by RalfGebhardt at January 18, 2018 12:19 PM

January 17, 2018

Oli Sennhauser

Advanced MySQL and MariaDB training in Cologne 2018

End of February, from February 26 to March 2 (5 days), FromDual offers an additional training for DBAs and DevOps: our most visited Advanced MySQL and MariaDB training.

This training is hold in the training facilities of the FromDual training partner GFU Cyrus GmbH in Cologne-Deutz (Germany).

There are already enough registrations so it is certain the training will take place. But there are still free places for at least 3 additional participants.

The training is in German.

You can find the training of this 5-day MySQL/MariaDB training here.

If you have any question please do not hesitate to contact us.

Taxonomy upgrade extras: 

by Shinguz at January 17, 2018 02:55 PM

Oracle releases MySQL security vulnerability fixes 2018-01

As in every quarter of the year Oracle has released yesterday its recommendation for the MySQL security updates. This is called, in Oracle terminology, Critical Patch Update (CPU) Advisory.

This CPU is published for all Oracle products. But FromDual is only interested in MySQL related topics. So let us concentrate on those.

This time 25 fixes with a maximum score of 8.1 (out of 10.0) were published.

6 of theses 25 vulnerabilities are exploitable remotely over the network without authentication (no user credentials required)!

The following MySQL products are affected:

  • MySQL Enterprise Monitor (3.3.6.3293 and before, 3.4.4.4226 and before, 4.0.0.5135 and before)
  • MySQL Connector/Net (6.9.9. and before, 6.10.4 and before)
  • MySQL Connector/ODBC (5.3.9. and before)
  • MySQL Server (5.5.58 and before, 5.6.38 and before, 5.7.19 and before)

It is recommended to upgrade your MySQL products to close the security vulnerabilities.

FromDual upgrade decision aid

Because such security updates are published quarterly and some of our customers have dozens to hundreds of MySQL installations this would end up in a never ending story where you are continuously upgrading MySQL database servers and other products.

This led to idea to create an upgrade decision aid to decide if you have to upgrade to this CPU or not.

The following questions can be asked:

  • How exposed is your database?
    Databases can be located in various network segments. It is not recommended to expose databases directly to the internet. Databases are either installed in demilitarized zones (DMZ) with no direct access from the internet or in the companies private network (only company employees should be able to access the database) or even specialized secure networks (only a limited number of specific employees can access this network).
  • How critical are your data?
    Some data are more interesting or critical, some data are less interesting or critical. Interesting data are: User data (user name and password), customer data (profiles, preferences, etc.), financial data (credit cards) and health care data (medical data). Systems containing such data are more critical than others. You can also ask: How sever is it if such data leak?
  • How broad is the user base able to access the database?
    How many employees do you have in your company? How many contractors do you have in your company? How many employees have physical access to the database server? How good is the mood of those people?
    How good are the user credentials to protect your database? Do you have shared passwords or no passwords at all? Do you have an account management (expiring old accounts, rotate passwords from time to time)?
    How much do you trust your users? Do you trust all your employees? Do you trust only admins? Or do you not even trust your admins?
  • How severe are the security vulnerabilities?
    You can define a threshold of severity of the vulnerabilities above you want to take actions. According to your criticality you can take actions for example as follows: Greater or equal than 7.5 if you have less critical data. Greater or equal than 6.0 if you have critical data.
  • Can the vulnerability be use from remote (over the network) and does it need a user authentication to exploit the vulnerability? What products (MySQL Enterprise Monitor, MySQL Server, MySQL Connectors) and what modules (Apache/Tomcat, .Net Connector, Partitioning, Stored Procedures, InnoDB, DDL, GIS, Optimizer, ODBC, Replication, DML, Performance Schema) are affected?

Depending on your readiness to take a risk you get now answers to decide if you have to take actions or not.

Some examples

  • Situation: Your database is exposed directly to the internet or you forgot to install some firewall rules to protect your MySQL port.
    Analysis: You are probably affected by CVE-2018-2696 and CVE-2017-3737 (score 5.9 and 7.5). So you passed the threshold for non-critical data (7.5) and nearly passed the threshold for critical data (6.0). These vulnerabilities allow attacks over the network without user authentication.
    Action: Immediate upgrade is recommended. Mid-term action: Install firewall rules to protect your MySQL to avoid access from remote and/or do not expose databases directly to the internet.
  • Situation: Your database is located in the intranet zone. You have slack user/password policies and you have many employees and also many contractors from foreign countries working on various projects. And you have very sensitive/interesting financial data stored in your database.
    Analysis: Many people, not all of them are really trusted, have network access to the database. It is quite possible that passwords have been shared or people have passwords for projects they are not working for any more. You are affected by nearly all of the vulnerabilities (network).
    Action: You should plan an upgrade soon. Mid-term action: Try to restrict access to the databases and implement some password policy rules (no shared passwords, password expiration, account locking etc.).
  • Situation: Your highly critical databases are located in a specially secured network and only applications, Linux admins and DBAs have access to this network. And you completely trust those people.
    Analysis: Your threshold is 6.0 and (unauthenticated) attack over the network is not possible. There are some vulnerabilities of which you are affected but the database is only accessed by an application. So those vulnerabilities cannot be exploited easily.
    Action: You possibly can ignore this CPU for the MySQL database this time. But you have a vulnerability in the .Net Connector (Connector/Net). If an attacker exploits the vulnerability on the Connector he possibly can get access to the data. So you have to upgrade the Connector of your application accessing the database.

If you follow the ideas of this aid you will probably have one or two upgrades a year. And this you should do anyway just to stay up to date...

See also Common Vulnerability Scoring System Version 3.0 Calculator.

Taxonomy upgrade extras: 

by Shinguz at January 17, 2018 10:27 AM

January 16, 2018

Valeriy Kravchuk

Fun with Bugs #60 - On Some Memory Leaks, Replication and Other Bugs Fixed in MySQL 5.7.21

Oracle had formally released MySQL 5.7.21 yesterday. I do not bother any more to study MySQL release notes carefully and completely, but during a quick review today I've noted several interesting items I'd like you to pay attention to.

I am historically interested in InnoDB implementation details, so I could not miss Bug #87619 - "InnoDB partition table will lock into the near record as a condition in the use ". This was a regression bug in 5.7+, probably caused by new implementation of partitioning in InnoDB.

Another interesting bug is Bug #86927 - "Renaming a partitioned table does not update mysql.innodb_table_stats.", by Jean-François Gagné. It was yet another bug in InnoDB's persistent statistics (that I truly hate). What makes it especially interesting to me, though, is that it's the first public bug report I noted that mentioned MySQL 9.0.0 release as a target for the fix:
"Fixed as of the upcoming 5.7.21, 8.0.4, 9.0.0 release"
So, it's clear that back in October 2017 Oracle had already got a separate branch for upcoming MySQL 9.0.x! It also probably means that MySQL 8.0.x GA is coming really soon.

There are bug reports that are worth reading for technical reasons, others - only if you want to get some fun. Bug #86607 - "InnoDB crashed when master thread evict dict_table_t object" is agood example that covers both cases. Good to know the crash is fixed, but, please, make sure to read all comments there.

In this release I've noted fixes to several public bugs reported by Shane Bester. The first one of them is Bug #86573 - "foreign key cascades use excessive memory". Check how he used memory instrumentation in Performance Schema to demonstrate the problem! In Bug #86482 - "innodb leaks memory, performance_schema file_instances #sql-ib3129987-252773.ibd", he used similar approach to show potential memory leak in the Performance Schema itself ! Yet another bug that mentions 9.0.0 as a target version for the fix, among others... 

Bug #78048 - "INNODB Full text Case sensitive not working", is here both because I recently started to notice problems related to InnoDB FULLTEXT indexing, again (first time was soon after it was introduced), and because it has an MTR  test case contributed by Sveta Smirnova.


XA transactions support had always been problematic in MySQL  (still "Verified" Bug #87526 by Sveta Smirnova is one of recent examples how incomplete or useless it can be, see also MDEV-14593). Check the following bugs fixed in MySQL 5.7.21 if you use XA transactions:
  • Bug #87393 - "xa rollback with wrong xid will be recorded into the binlog". It was reported by HongXiang Jiang, who had also contributed a patch.
  • Bug #83295 - "replication error occurs, use xa transaction(one phase)". Yet another XA transactions problem reported by Hiroyuki Itoh and then confirmed by many affected users. Nice to see it fixed.
There are many fixes in MySQL 5.7.21 related to memory leaks. Two bug reports of this kind were from Przemyslaw Malkowski:
  • Bug #85371 - "Memory leak in multi-source replication when binlog_rows_query_log_events=1". Again, memory instrumentation of Performance Schema was used to demonstrate the problem. Vlad Lesin, also from Percona, contributed the patch for this bug.
  • Bug #85251 - "Memory leak in master-master GTID replication with sync_relay_log_info". Here Vlad Lesin, who had contributed the patch, also used Massif for the detailed analysis.
To summarize, I start to miss memory instrumentation in Performance Schema in MariaDB 10.x... This is a really useful feature.

I usually care about optimizer bugs, and these two attracted my attention:
  • Bug #87207 - "select distinct with secondary key for 'Using index for group-by' bad results". This nice optimizer regression bug was found by Shane Bester. As a workaround, while you do not use 5.7.21, you can try to set optimizer_switch='use_index_extensions=off'. I'd keep it that way by default...
  • Bug #72854 - "Extremely slow performance with outer joins and join buffer". I am happy to see this old optimizer bug reported by Sergey Petrunya from MariaDB finally fixed.
You can find a lot more details, including usual references to MySQL bug reports that are still private, in the Release Notes. Keep reading and consider upgrade :)

by Valeriy Kravchuk (noreply@blogger.com) at January 16, 2018 05:58 PM

Jean-Jerome Schmidt

MySQL on Docker - How to Containerize Your Database - New Whitepaper

Severalnines is happy to announce that our new whitepaper “MySQL on Docker - How to Containerize Your Database” is now available to download for free!

While the idea of containers has been around since the early days of Unix, Docker made waves in 2013 when it hit the market with its innovative solution. Docker allows you to add your stacks and applications to containers where they share a common operating system kernel. This lets you have a lightweight virtualized system with almost zero overhead. Docker also lets you bring containers up or down in seconds, making for rapid deployment of your stack.

Download whitepaper

Severalnines has been experimenting with and writing about how to utilize Docker for MySQL in our MySQL on Docker Blog Series since 2014.

This new white paper is the culmination of years of work by our team trying to understand how best to deploy and manage MySQL on Docker while utilizing the advanced monitoring and management features found in ClusterControl.

The topics covered in this white paper are...

  • An Introduction to Docker
  • MySQL Docker Images
  • Networking in Docker
  • Understanding MySQL Containers & Volume
  • Monitoring and Management of MySQL in Docker
    • Docker Security
    • Backup and Restores
  • Running ClusterControl on Docker

If your organization is or plans on taking advantage of the latest in Docker container technology in conjunction with their open source MySQL databases, this whitepaper will help you better understand what you need to do to get started.

ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

ClusterControl on Docker

ClusterControl provides advanced management and monitoring functionality to get your MySQL replication and clustered instances up-and-running using proven methodologies that you can depend on to work. Used in conjunction with other orchestration tools for deployment to the containers, ClusterControl makes managing your open source databases easy with point-and-click interfaces and no need to have specialized knowledge about the technology.

ClusterControl delivers on an array of features to help manage and monitor your open source database environments:

  • Management & Monitoring: ClusterControl provides management features to repair and recover broken nodes, as well as test and automate MySQL upgrades.
  • Advanced Monitoring: ClusterControl provides a unified view of all MySQL nodes and clusters across all your data centers and lets you drill down into individual nodes for more detailed statistics.
  • Automatic Failure Detection and Handling: ClusterControl takes care of your replication clusters health. If a master failure is detected, ClusterControl automatically promotes one of the available slaves to ensure your cluster is always up.

Learn more about how ClusterControl can enhance performance here or pull the Docker Image here.

by Severalnines at January 16, 2018 11:31 AM

MariaDB Foundation

Protected branches – ensuring code quality in git

In order to ensure that new (or changed) code does not break anything, there is an extensive test suite that is run to catch regressions during MariaDB Server development. Developers are expected to run the test suite locally and, after pushing the code to the remote repository, also check that the more extensive tests run […]

The post Protected branches – ensuring code quality in git appeared first on MariaDB.org.

by Otto Kekäläinen at January 16, 2018 07:21 AM

MariaDB AB

Introducing MariaDB MaxScale 2.2.1 Beta - automation, security and more!

Introducing MariaDB MaxScale 2.2.1 Beta - automation, security and more! Dipti Joshi Mon, 01/15/2018 - 21:49

We are happy to announce that  2.2.1 beta of MariaDB MaxScale, the next generation database proxy for MariaDB is now available. Beta is an important time in our release and we encourage you to download this release today!

The goal of this MariaDB MaxScale release is self-healing automation, hardened database security, preparing for new capabilities for the upcoming MariaDB Server 10.3 release, while at the same time, making it easier to manage.

MariaDB MaxScale 2.2.1 introduces the following key new capabilities:

  • Automatic failover, manual failover and switchover for MariaDB Master/Slave Clusters:

    • MaxScale can now automatically select and promote a slave to master when the master node fails.

    • MaxScale now provides the user to administratively switchover a slave to master.

    • MaxScale allows a suitable server to be added to a running Master/Slave cluster as slave.

  • MariaDB Server feature support:

    • Proxy protocol support for backend connections, a MariaDB Server 10.3 feature. This makes it easy to configure authorized use between the Client, MaxScale and Server.

    • PL/SQL Compatibility syntax support for upcoming MariaDB Server 10.3 support.

    • Support of window functions and CTEs processing.

    • Binlog router supports MariaDB 10 GTID at both the master and slave side.

  • Management Interface:

    • REST-API for obtaining information about and managing the resources of MaxScale.

    • MaxCtrl, a new command line client for administering MaxScale implemented using the REST-API, for managing an individual MaxScale as well as a cluster of MaxScales

  • Security:

  • Reliability:

    • MariaDB Server states are persisted, so in the case of a crash and a restart of MaxScale, it has the correct server state, quicker.

    • Monitor scripts are executed synchronously, so they can safely perform actions that change the server states.

  • Prepared statements are now parsed and the execution of read only ones will be routed to slaves.

  • KILL CONNECTION can now be used through MaxScale.

The release notes for MariaDB MaxScale 2.2.1 can be found here and the list of bugs fixed can be found in the release notes. Binaries for MaxScale 2.2.1 Beta are available for download here. MaxScale documentation can be found in the Knowledge Base. In case you want to build the binaries yourself, the source can be found on GitHub, tagged with maxscale-2.2.1.

We are very excited about this release and look forward to hearing from our users about their feedback and experiences. If you have any questions or feedback, please email me dipti.joshi@mariadb.com or reach out through our community group.

We are happy to announce that  2.2.1 beta of MariaDB MaxScale, the next generation database proxy for MariaDB is now available. Beta is an important time in our release and we encourage you to download this release today!

Login or Register to post comments

by Dipti Joshi at January 16, 2018 02:49 AM

January 15, 2018

Jean-Jerome Schmidt

ClusterControl Tips & Tricks: Securing your MySQL Installation (Updated)

Requires ClusterControl 1.2.11 or later. Applies to MySQL based clusters.

During the life cycle of Database installation it is common that new user accounts are created. It is a good practice to once in a while verify that the security is up to standards. That is, there should at least not be any accounts with global access rights, or accounts without password.

Using ClusterControl, you can at any time perform a security audit.

In the User Interface go to Manage > Developer Studio. Expand the folders so that you see s9s/mysql/programs. Click on security_audit.js and then press Compile and Run.

If there are problems you will clearly see it in the messages section:

Enlarged Messages output:

Here we have accounts that can connect from any hosts and accounts which do not have a password. Those accounts should not exist in a secure database installation. That is rule number one. To correct this problem, click on mysql_secure_installation.js in the s9s/mysql/programs folder.

Click on the dropdown arrow next to Compile and Run and press Change Settings. You will see the following dialog and enter the argument “STRICT”:

Then press Execute. The mysql_secure_installation.js script will then do on each MySQL database instance part of the cluster:

  1. Delete anonymous users
  2. Dropping 'test' database (if exists).
  3. If STRICT is given as an argument to mysql_secure_installation.js it will also do:
    • Remove accounts without passwords.

In the Message box you will see:

The MySQL database servers part of this cluster have now been secured and you have reduced the risk of compromising your data.

You can re-run security_audit.js to verify that the actions have had effect.

Happy Clustering!

PS.: To get started with ClusterControl, click here!

by krzysztof at January 15, 2018 08:02 AM

January 11, 2018

Jean-Jerome Schmidt

Ten Tips on How to Achieve MySQL and MariaDB Security

Security of data is a top priority these days. Sometimes it’s enforced by external regulations like PCI-DSS or HIPAA, sometimes it’s because you care about your customers’ data and your reputation. There are numerous aspects of security that you need to keep in mind - network access, operating system security, grants, encryption and so on. In this blog post, we’ll give you 10 tips on what to look at when securing your MySQL or MariaDB setup.

1. Remove users without password

MySQL used to come with a set of pre-created users, some of which can connect to the database without a password or, even worse, anonymous users. This has changed in MySQL 5.7 which, by default, comes only with a root account that uses the password you choose at installation time. Still, there are MySQL installations which were upgraded from previous versions and these installations keep the legacy users. Also, MariaDB 10.2 on Centos 7 comes with anonymous users:

MariaDB [(none)]> select user, host, password from mysql.user where user like '';
+------+-----------------------+----------+
| user | host                  | password |
+------+-----------------------+----------+
|      | localhost             |          |
|      | localhost.localdomain |          |
+------+-----------------------+----------+
2 rows in set (0.00 sec)

As you can see, those are limited only to access from localhost but regardless, you do not want to have users like that. While their privileges are limited, they still can run some commands which may show more information about the database - for example, the version may help identify further vectors of attack.

[root@localhost ~]# mysql -uanonymous_user
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 19
Server version: 10.2.11-MariaDB MariaDB Server
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> SHOW GRANTS\G
*************************** 1. row ***************************
Grants for @localhost: GRANT USAGE ON *.* TO ''@'localhost'
1 row in set (0.00 sec)
MariaDB [(none)]> \s
--------------
mysql  Ver 15.1 Distrib 10.2.11-MariaDB, for Linux (x86_64) using readline 5.1
Connection id:        19
Current database:
Current user:        anonymous_user@localhost
SSL:            Not in use
Current pager:        stdout
Using outfile:        ''
Using delimiter:    ;
Server:            MariaDB
Server version:        10.2.11-MariaDB MariaDB Server
Protocol version:    10
Connection:        Localhost via UNIX socket
Server characterset:    latin1
Db     characterset:    latin1
Client characterset:    utf8
Conn.  characterset:    utf8
UNIX socket:        /var/lib/mysql/mysql.sock
Uptime:            12 min 14 sec
Threads: 7  Questions: 36  Slow queries: 0  Opens: 17  Flush tables: 1  Open tables: 11  Queries per second avg: 0.049
--------------

Please note that users with very simple passwords are almost as insecure as users without any password. Passwords like “password” or “qwerty” are not really helpful.

2. Tight remote access

First of all, remote access for superusers - this is taken care of by default when installing the latest MySQL (5.7) or MariaDB (10.2) - only local access is available. Still, it’s pretty common to see superusers being available for various reasons. The most common one, probably because the database is managed by humans who want to make their job easier, so they’d add remote access to their databases. This is not a good approach as remote access makes it easier to exploit potential (or verified) security vulnerabilities in MySQL - you don’t need to get a connection to the host first.

Another step - make sure that every user can connect to MySQL only from specific hosts. You can always define several entries for the same user (myuser@host1, myuser@host2), this should help to reduce a need for wildcards (myuser@’%’).

3. Remove test database

The test database, by default, is available to every user, especially to the anonymous users. Such users can create tables and write to them. This can potentially become a problem on its own - any writes would add some overhead and reduce database performance. Currently, after the default instalation, only MariaDB 10.2 on Centos 7 is affected by this - Oracle MySQL 5.7 and Percona Server 5.7 do not have the ‘test’ schema available.

[root@localhost ~]# mysql -uanonymous_user
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 13
Server version: 10.2.11-MariaDB MariaDB Server
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> SHOW GRANTS\G
*************************** 1. row ***************************
Grants for @localhost: GRANT USAGE ON *.* TO ''@'localhost'
1 row in set (0.00 sec)
MariaDB [(none)]> USE test;
Database changed
MariaDB [test]> CREATE TABLE testtable (a INT);
Query OK, 0 rows affected (0.01 sec)
MariaDB [test]> INSERT INTO testtable VALUES (1), (2), (3);
Query OK, 3 rows affected (0.01 sec)
Records: 3  Duplicates: 0  Warnings: 0
MariaDB [test]> SELECT * FROM testtable;
+------+
| a    |
+------+
|    1 |
|    2 |
|    3 |
+------+
3 rows in set (0.00 sec)

Of course, it may still happen that your MySQL 5.7 has been upgraded from previous versions in which the ‘test’ schema was not removed - you should take care of this and check if you have it created.

4. Obfuscate access to MySQL

It is well known that MySQL runs on port 3306, and its superuser is called ‘root’. To make things harder, it is quite simple to change this. To some extent, this is an example of security through obscurity but it may at least stop automated attempts to get access to the ‘root’ user. To change port, you need to edit my.cnf and set ‘port’ variable to some other value. As for users - after MySQL is installed, you should create a new superuser (GRANT ALL … WITH GRANT OPTION) and then remove existing ‘root@’ accounts.

5. Network security

Ideally, MySQL would be not available through the network and all connections would be handled locally, through the Unix socket. In some setups, this is possible - in that case you can add the ‘skip-networking’ variable in my.cnf. This will prevent MySQL from using any TCP/IP communication, only Unix socket would be available on Linux (Named pipes and shared memory on Windows hosts).

Most of the time though, such tight security is not feasible. In that case you need to find another solution. First, you can use your firewall to allow traffic only from specific hosts to the MySQL server. For instance, application hosts (although they should be ok with reaching MySQL through proxies), the proxy layer, and maybe a management server. Other hosts in your network probably do not need direct access to the MySQL server. This will limit possibilities of attack on your database, in case some hosts in your network would be compromised.

If you happen to use proxies which allow regular expression matching for queries, you can use them to analyze the SQL traffic and block suspicious queries. Most likely your application hosts shouldn’t run “DELETE * FROM your_table;” on a regular basis. If it is needed to remove some data, it can be executed by hand, locally, on the MySQL instance. You can create such rules using something like ProxySQL: block, rewrite, redirect such queries. MaxScale also gives you an option to block queries based on regular expressions.

6. Audit plugins

If you are interested in collecting data on who executed what and when, there are several audit plugins available for MySQL. If you use MySQL Enterprise, you can use MySQL Enterprise Audit which is an extension to MySQL Enterprise. Percona and MariaDB also have their own version of audit plugins. Lastly, McAfee plugin for MySQL can also be used with different versions of MySQL. Generally speaking, those plugins collect more or less the same data - connect and disconnect events, queries executed, tables accessed. All of this contains information about which user participated in such event, from what host it logged from, when did it happen and so on. The output can be XML or JSON, so it’s much easier to parse it than parsing general log contents (even though the data is rather similar). Such output can also be sent to syslog and, further, some sort of log server for processing and analysis.

7. Disable LOAD DATA LOCAL INFILE

If both server and client has the ability to run LOAD DATA LOCAL INFILE, a client will be able to load data from a local file to a remote MySQL server. This, potentially, can help to read files the client has access to - for example, on an application server, one could access any file that the HTTP server has access to. To avoid it, you need to set local-infile=0 in the my.cnf

8. File privileges

You have to keep in mind that MySQL security also depends on the operating system setup. MySQL stores data in the form of files. The MySQL server writes plenty of information to logs. Sometimes this information contains data - slow query log, general log or binary log, for example. You need to make sure that this information is safe and accesible only to users who have to access it. Typically it means that only the root and the user under whose rights MySQL is running, should have access to all MySQL-related files. Most of the time it’s a dedicated user called ‘mysql’. You should check MySQL configuration files and all the logs generated by MySQL and verify that they are not readable by other users.

ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

9. SSL and Encryption of Data in Transit

Preventing people from accessing configuration and log files is one thing. The other issue is to make sure data is securely transferred over the network. With an exception of setups where all the clients are local and use Unix socket to access MySQL, in majority of cases, data which forms a result set for a query, leaves the server and is transferred to the client over the network. Data can also be transferred between MySQL servers, for example via standard MySQLreplication or within a Galera cluster. Network traffic can be sniffed, and through those means, your data would be exposed.

To prevent this from happening, it is possible to use SSL to encrypt traffic, both server and client-side. You can create an SSL connection between a client and a MySQL server. You can also create an SSL connection between your master and your slaves, or between the nodes of a Galera cluster. This will ensure that all data that is transferred is safe and cannot be sniffed by an attacker who gained access to your network.

The MySQL documentation covers in detail how to setup SSL encryption. If you find it too cumbersome, ClusterControl can help you deploy a secure environment for MySQL replication or Galera cluster in a couple of clicks:

10. Encryption of Data at Rest

Securing data in transit using SSL encryption only partially solves the problem. You need to take care also of data at rest - all the data that is stored in the database. Data at rest encryption can also be a requirement for security regulations like HIPAA or PCI DSS. Such encryption can be implemented on multiple levels - you can encrypt the whole disk on which the files are stored. You can encrypt only the MySQL database through functionality available in the latest versions of MySQL or MariaDB. Encryption can also be implemented in the application, so that it encrypts the data before storing it in the database. Every option has its pros and cons: disk encryption can help only when disks are physically stolen, but the files would not be encrypted on a running database server. MySQL database encryption solves this issue, but it cannot prevent access to data when the root account is compromised. Application level encryption is the most flexible and secure, but then you lose the power of SQL - it’s pretty hard to use encrypted columns in WHERE or JOIN clauses.

All flavors of MySQL provide some sort of data at rest encryption. Oracle’s MySQL uses Transparent Data Encryption to encrypt InnoDB tablespaces. This is available in the commercial MySQL Enterprise offering. It provides an option to encrypt InnoDB tablespaces, other files which also store data in some form (for example, binary logs, general log, slow query log) are not encrypted. This allows the toolchain (MySQL Enterprise Backup but also xtrabackup, mysqldump, mysqlbinlog) to work correctly with such setup.

Starting from MySQL 5.7.11, the community version of MySQL also got support for InnoDB tablespace encryption. The main difference compared to the enterprise offering is the way the keys are stored - keys are not located in a secure vault, which is required for regulatory compliance. This means that starting from Percona Server 5.7.11, it is also possible to encrypt InnoDB tablespace. In the recently published Percona Server 5.7.20, support for encrypting binary logs has been added. It is also possible to integrate with Hashicorp Vault server via a keyring_vault plugin, matching (and even extending - binary log encryption) the features available in Oracle’s MySQL Enterprise edition.

MariaDB added support for data encryption in 10.1.3 - it is a separate, enhanced implementation. It gives you the possibility to not only encrypt InnoDB tablespaces, but also InnoDB log files. As a result, data is more secure but some of the tools won’t work in such configuration. Xtrabackup will not work with encrypted redo logs - MariaDB created a fork, MariaDB Backup, which adds support for MariaDB encryption. There are also issues with mysqlbinlog.

No matter which MySQL flavor you use, as long as it is a recent version, you would have options to implement data at rest encryption via the database server, making sure that your data is additionally secured.

Securing MySQL or MariaDB is not trivial, but we hope these 10 tips will help you along the way.

by krzysztof at January 11, 2018 10:28 AM

January 10, 2018

MariaDB AB

MariaDB ColumnStore Distributed User Defined Aggregate Functions

MariaDB ColumnStore Distributed User Defined Aggregate Functions david.hall@mar… Wed, 01/10/2018 - 11:31

MariaDB ColumnStore 1.1 introduces the Distributed User Defined Aggregate Functions (UDAF) C++ API. MariaDB Server has supported UDAF (a C API) for a while, but now we have extended it to the ColumnStore Engine. This new feature allows anyone to create aggregate functions of arbitrary complexity for distributed execution in the ColumnStore Engine. These functions can also be used as Analytic (Window) functions just like any built in aggregate. You should have a working understanding of C++ to use this API.

The UDAF API supports an arbitrary number of parameters to be defined for your Aggregate function. Version 1.1 of ColumnStore supports one parameter and will be enhanced to support any number of parameters in the next version.

For example, defining MEDIAN() is allowed, but PERCENTILE_CONT(, ) will be enabled in the next version. However, there are some workarounds for the short term, such as using a distributed MEDIAN which provides equivalent functionality to PERCENTILE_CONT(0.5). In this his example, MEDIAN can be extended to support other percentile values. If you want to support a 90th percentile function, then a PERCENTILE90() function can be implemented currently.

So, how does this work? MariaDB ColumnStore has had a distributed aggregate function capability since the start. With the 1.1 release, we’ve added new functionality that allows the engine to recognize when a UDAF has been called and perform callbacks at each level of execution. The code you write mustdefine the data structures and the work to be performed on those structures.

It’s relatively straight forward. There are two classes you need to become familiar with -- mcsv1Context and mcsv1_UDAF. mcsv1Context are the classes that holds the state of your function during execution. mcsv1_UDAF is the class you extend to write your code.

To write a UDAF, this requires extending the class mcsv1_UDAF and implementing a few functions. In many cases, these implementations are straightforward, but it really depends on what you’re trying to do. For use as an aggregate, some functions are called on the UM, some on the PM. For example, the UDAF ssq.cpp is a simple implementation that shows the basic implementation of each function.

UDAF.png

For use as Analytic functions, all calls are on the UM.

UDAnF.png

​In addition, you must write the same exact function in the MariaDB UDAF C API. This is required and can be a simple stub or a complete implementation, depending on whether you want your function to work as an aggregate for other engines. But, it is needed to tell the parser that your function exists.

Since memory must be allocated at each node for the work being performed there, MariaDB ColumnStore handles when and where memory is allocated. You may choose to provide a method to do that allocation which the engine calls when it needs to.

You may have a need for some complex data structure, hash table, vectors or other subobjects. In this situation, you need to become familiar with the Complex Data Model as described in the udaf_sdk Documentation. There is no limit to the complexity of the memory model you choose. It is important that you create a UserData derived class that can serialize and un-serialize itself. It’s destructor must clean up all allocated memory.

If all you need is a simple data structure, you may forgo all the memory allocation steps and rely on the base UserData class. Its default functionality is to allocate a fixed block of memory and to stream that block as a single binary value. You need do nothing except set the amount of memory in the init() method and overlay your structure in each callback method.

MariaDB ColumnStore 1.1 doesn’t support dynamic loading of plugins, so your UDAF must be compiled and linked in the MariaDB ColumnStore code tree in the ./utils.udfsdk directory. You must compile and link it into libudfsdk.so.1.1.0 along with all the other user defined functions. This library must be placed into the mariadb/columnstore/lib directory of each node. The MariaDB C UDAF code must be compiled and linked into libudf_mysql.so.1.0.0 and placed into the same place. There’s a symlink already there for mysqld to find it.

Then to activate your UDAF, in a mysql client, issue a command similar to:
CREATE AGGREGATE FUNCTION median returns REAL soname 'libudf_mysql.so'

In future blogs, I’ll delve deep into each step needed to create and use a UDAF.

User Defined Aggregates open up a whole new avenue to extract value from analytic data. We hope you enjoy using this new tool! MariaDB ColumnStore 1.1 is available for download as part of MariaDB AX, an enterprise open source solution for modern data analytics and data warehousing.

MariaDB ColumnStore 1.1 introduces the Distributed User Defined Aggregate Functions (UDAF) C++ API. MariaDB Server has supported UDAF (a C API) for a while, but now we have extended it to the ColumnStore Engine. This new feature allows anyone to create aggregate functions of arbitrary complexity for distributed execution in the ColumnStore Engine. These functions can also be used as Analytic (Window) functions just like any built in aggregate. You should have a working understanding of C++ to use this API.

Login or Register to post comments

by MariaDB AB (david.hall@mariadb.com) at January 10, 2018 04:31 PM

Jean-Jerome Schmidt

How to Secure Your Open Source Databases with ClusterControl

Security is one of the most important aspects of running a database. Whether you are a developer or a DBA, if you are managing the database, it is your responsibility to safeguard your data and protect it from any kind of unauthorized access. The unfortunate fact is that many organizations do not protect their data, as we’ve seen from the new wave of MongoDB ransomware attacks in September 2017. We had earlier published a blog on how to secure MongoDB databases.

In this blog post, we’ll have a look into how to secure your databases using ClusterControl. All of the features described here are available in version 1.5.1 of ClusterControl (released on December 23, 2017). Please note that some features are only available for certain database types.

Backup Encryption

ClusterControl 1.5.1 introduced a new feature called backup encryption. All encrypted backups are marked with a lock icon next to it:

You can use this feature on all backup methods (mysqldump, xtrabackup, mongodump, pg_dump) supported by ClusterControl. To enable encryption, simply toggle on the "Enable Encryption" switch when scheduling or creating the backup. ClusterControl automatically generates a key to encrypt the backup. It uses AES-256 (CBC) encryption algorithm and performs the encryption on-the-fly on the target server. The following command shows an example of how ClusterControl performs a mysqldump backup:

$ mysqldump --defaults-file=/etc/my.cnf --flush-privileges --hex-blob --opt --no-create-info --no-data --triggers --routines --events --single-transaction --skip-comments --skip-lock-tables --skip-add-locks --databases db1 | gzip -6 -c | openssl enc -aes-256-cbc -pass file:/var/tmp/cmon-094508-e0bc6ad658e88d93.tmp | socat - TCP4:192.168.55.170:9999'

You would see the following error if you tried to decompress an encrypted backup without decrypting it first with the proper key:

$ gunzip mysqldump_2018-01-03_175727_data.sql.gz
gzip: mysqldump_2018-01-03_175727_data.sql.gz: not in gzip format

The key is stored inside the ClusterControl database, and can be retrieved from the cmon_backup.metadata file for a particular backup set. It will be used by ClusterControl when performing restoration. Encrypting backups is highly recommended, especially when you want to secure your backups offsite like archiving them in the cloud.

MySQL/PostgreSQL Client-Server Encryption

Apart from following the recommended security steps during deployment, you can increase the reliability of your database service by using client-server SSL encryption. Using ClusterControl, you can perform this operation with simple point and click:

You can then retrieve the generated keys and certificates directly from the ClusterControl host under /var/lib/cmon/ca path to establish secure connections with the database clients. All the keys and certificates can be managed directly under Key Management, as described further down.

Database Replication Encryption

Replication traffic within a Galera Cluster can be enabled with just one click. ClusterControl uses a 2048-bit default key and certificate generated on the ClusterControl node, which is transferred to all the Galera nodes:

A cluster restart is necessary. ClusterControl will perform a rolling restart operation, taking one node at a time. You will see a green lock icon next to the database server (Galera indicates Galera Replication encryption, while SSL indicates client-server encryption) in the Hosts grid of the Overview page once encryption is enabled:

All the keys and certificates can be managed directly under Key Management, as described further down.

ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

Key Management

All the generated keys and certificates can be managed directly from the ClusterControl UI. Key Management allows you to manage SSL certificates and keys that can be provisioned on your clusters:

If the certificate has expired, you can simply use the UI to generate a new certificate with proper key and Certificate Authority (CA), or import an existing key and certificate into ClusterControl host.

Security Advisors

Advisors are mini-programs that run in ClusterControl. They perform specific tasks and provide advice on how to address issues in areas such as performance, security, log management, configuration, storage space and others. Each advisor can be scheduled like a cron job, and run as a standalone executable within the ClusterControl UI. It can also be run via the ClusterControl 's9s' command line client.

ClusterControl enables two security advisors for MySQL-based systems:

  • Access from any host ('%') - Identifies all users that use a wildcard host from the mysql system table, and lets you have more control over which hosts are able to connect to the servers.
  • Check number of accounts without a password - Identifies all users who do not have a password in the mysql system table.

For MongoDB, we have the following advisors:

  • MongoDB authentication enabled - Check whether the MongoDB instance is running with authentication mode enabled.
  • Authorization check - Check whether MongoDB users are authorized with too permissive role for access control.

For more details on how does ClusterControl performs the security checks, you can look at the advisor JavaScript-like source code under Manage -> Developer Studio. You can see the execution results from the Advisors page:

Multiple Network Interfaces

Having multiple NICs on the database hosts allows you to separate database traffic from management traffic. One network is used by the database nodes in order to communicate to each other, and this network is not exposed to any public network. The other network is used by ClusterControl, for management purposes. ClusterControl is able to deploy such a multi-network setup. Consider the following architecture diagram:

To import the above database cluster into ClusterControl, one would specify the primary IP address of the database hosts. Then, it is possible to choose the management network as well as the data network:

ClusterControl can also work in an environment without Internet access, with the databases being totally isolated from the public network. The majority of the features will work just fine. If the ClusterControl host is configured with Internet, it is also capable of cloning the database vendor's repository for the internet-less database servers. Just go to Settings (top menu) -> Repositories -> Create New Repository and set the options to fit the target database server environment:

The mirroring may take about 10 to 20 minutes depending on the internet connection, you will see the new item in the list later on. You can then pick this repository instead when scaling or deploying a new cluster, without the need for the database hosts to have any Internet connection (note that the operating system’s offline repository should be in place as well).

MySQL Users Management

The MySQL privilege system ensures that all users can perform only the operations they are allowed to. Granting is critical as you don't want to give all users complete access to your database, but you need users to have the necessary permissions to run queries and perform daily tasks.

ClusterControl provides an interactive user interface to manage the database schemas and privileges. It unifies the accounts on all MySQL servers in the cluster and simplifies the granting process. You can easily visualize the database users, so you avoid making mistakes.

As you can see in the above screenshot, ClusterControl greyed out unnecessary privileges if you only want to grant a user to a database (shopdb). "Require SSL?" is only enabled if the client/server SSL encryption is enabled while the administration privilege checkboxes are totally disabled if a specific database is defined. You can also inspect the generated GRANT statement at the bottom of the wizard, to see the statement that ClusterControl will execute to create this user. This helper looks pretty simple, but creating users and granting privileges can be error-prone.

ClusterControl also provides a list of inactive users for all database nodes in the cluster, showing off the accounts that have not been used since the last server restart:

This alerts the administrator for unnecessary accounts that exist, and that could potentially harm the server. The next step is to verify if the accounts are no longer active, and you can simply use the "Drop Selected User" option in order to remove them. Make sure you have enough database activity to ensure the list generated by ClusterControl is accurate. The longer the server uptime, the better.

Always Keep Up-to-date

For production use, it’s highly recommended for you to install the database-related packages from the vendor’s repository. Don’t rely on the default operating system repository, where the packages are usually outdated. If you are running in a cluster environment like Galera Cluster, or even MySQL Replication, you always have the choice to patch the system with minimal downtime.

ClusterControl supports automatic minor version rolling upgrade for MySQL/MariaDB with a single click. Just go to Manage -> Upgrades -> Upgrade and choose the appropriate major version for your running cluster. ClusterControl will then perform the upgrade, on one node at a time. The node will be stopped, then software will be updated, and then the node will be started again. If a node fails to upgrade, the upgrade process is aborted and the admin is notified. Upgrades should only be performed when there is as little traffic as possible on the cluster.

Major versions upgrades (e.g, from MySQL 5.6 to MySQL 5.7) are intentionally not automated. Major upgrades usually require uninstallation of the existing packages, which is a risky task to automate. Careful planning and testing is necessary for such kind of upgrades.

Database security is an important aspect of running your database in production. From all the incidents we frequently read about in the news (and there are probably many others that are not publicized), it is clear that there are groups busy out there with bad intentions. So, make sure your databases are well protected.

by ashraf at January 10, 2018 09:56 AM

January 09, 2018

MariaDB AB

MariaDB MaxScale 2.1 and SSL Certificates

MariaDB MaxScale 2.1 and SSL Certificates Wagner Bianchi Tue, 01/09/2018 - 13:44

MariaDB MaxScale has becoming increasingly popular as a database proxy, adopted by users that would like to take advantage of a good strategy for scaling out databases and data infrastructure. As with any database environment, it is important to make the environment safe and to adopt the top industry best practices. Most MariaDB MaxScale users have or will have MaxScale handling traffic to database instances/backends in a wan, where servers can be added to MariaDB’s intelligent database proxy and based on the configurations, traffic is routed to those servers. In some cases, man-in-the-middle and other attack strategies are used to intercept information while data is being replicated and, while connections are routed to the backend databases.

This blog post will explore the setup of an environment using self-signed OpenSSL certificates to make it safe enough to replicate data between the multiple backend database servers and also, we’ll show you how to setup the communication between MariaDB MaxScale and the backend.

The following are the servers and version we’re using on this blog:

  • 4 VMs vagrant-wise created:
    • 1 MariaDB MaxScale;
    • 1 MariaDB Server as Master;
    • 2 MariaDB Server as Replica;
  • CentOS 7.3 as the operating system;
  • MariaDB Server 10.2.10;
  • MariaDB MaxScale 2.1.10;

For this blog, I assume you already have the servers configured and replicating (one master and two replicas).

MariaDB MaxScale will look like this below at the end of this blog:

[root@maxscale ~]# maxadmin list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server             | Address         | Port  | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
prod_mariadb01     | 192.168.50.11   |  3306 |           0 | Master, Running
prod_mariadb02     | 192.168.50.12   |  3306 |           0 | Slave, Running
prod_mariadb03     | 192.168.50.13   |  3306 |           0 | Slave, Running
-------------------+-----------------+-------+-------------+--------------------

If you're following this tutorial, make sure you setup your servers via the MariaDB official repository to have access to the software we will need for set up.

#: setup MariaDB Official repository
[root@box01 ~]# curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | sudo bash
[info] Repository file successfully written to /etc/yum.repos.d/mariadb.repo.
[info] Adding trusted package signing keys...
[info] Succeessfully added trusted package signing keys.

Generating the Self-Signed Certificates

The place to start with this is to generate your self-signed OpenSSL certificates, but, if you would like to acquire a certificate from any of the existing entities that will sign the certificate for you, that’s fine as well. Here, I’m going through the creation of certificates with OpenSSL, present on most of the Linux Distributions by the default and then, I’m going to use that. Below you can find the command to generate your certificates, the same as I used to generate the certificates at /etc/my.cnf.d/certs/. One detail here is that you won’t see this directory on the MariaDB MaxScale host, so, you will need to create that directory and move certs there.

[root@maxscale ~]# mkdir -pv /etc/my.cnf.d/certs/
mkdir: created directory ‘/etc/my.cnf.d/certs/’

[root@box01 ~]# mkdir -pv /etc/my.cnf.d/certs/
mkdir: created directory ‘/etc/my.cnf.d/certs/’

[root@box02 ~]# mkdir -pv /etc/my.cnf.d/certs/
mkdir: created directory ‘/etc/my.cnf.d/certs/’

[root@box03 ~]# mkdir -pv /etc/my.cnf.d/certs/
mkdir: created directory ‘/etc/my.cnf.d/certs/’

I created a directory on the MariaDB MaxScale server host, moved my prompt to /etc/my.cnf.d/certs/ and then, created the certificates using the below commands.

#: generate the ca-key
$ openssl genrsa 2048 > ca-key.pem

#: server certs
$ openssl req -new -x509 -nodes -days 9999 -key ca-key.pem > ca-cert.pem
$ openssl req -newkey rsa:2048 -days 1000 -nodes -keyout server-key.pem > server-req.pem
$ openssl x509 -req -in server-req.pem -days 3600 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 -out server-cert.pem

#: client certs
$ openssl req -newkey rsa:2048 -days 3600 -nodes -keyout client-key.pem -out client-req.pem
$ openssl rsa -in client-key.pem -out client-key.pem
$ openssl x509 -req -in client-req.pem -days 3600 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 -out client-cert.pem

#: verify generated certificates
$ openssl verify -CAfile ca-cert.pem server-cert.pem client-cert.pem
server-cert.pem: OK
client-cert.pem: OK

One thing you should be aware of is if the last part doesn’t go well and the certificates verification don’t give you an OK is that you need to have different names for the CN or Common Names. The error that appeared sometimes is like the one below:

#: execution the SSL certificates verification
$ openssl verify -CAfile ca-cert.pem server-cert.pem client-cert.pem
server-cert.pem: C = BR, ST = MG, L = BH, O = WBC, OU = WB, CN = WB, emailAddress = me@all.com
error 18 at 0 depth lookup:self signed certificate
OK
client-cert.pem: C = BR, ST = MG, L = BH, O = WBC, OU = WB, CN = WB, emailAddress = me@all.com
error 18 at 0 depth lookup:self signed certificate
OK

After finishing the certificate's creation successfully and then, pass through the verification, as shown above, you will have the following files at /etc/my.cnf.d/certs/:

#: listing servers on MariaDB Maxscale server host
[root@maxscale ~]# ls -lh /etc/my.cnf.d/certs/
total 32K
-rw-r--r-- 1 root root 1.4K Nov  5 11:08 ca-cert.pem
-rw-r--r-- 1 root root 1.7K Nov  5 11:07 ca-key.pem
-rw-r--r-- 1 root root 1.3K Nov  5 11:11 client-cert.pem
-rw-r--r-- 1 root root 1.7K Nov  5 11:11 client-key.pem
-rw-r--r-- 1 root root 1.1K Nov  5 11:10 client-req.pem
-rw-r--r-- 1 root root 1.3K Nov  5 11:09 server-cert.pem
-rw-r--r-- 1 root root 1.7K Nov  5 11:09 server-key.pem
-rw-r--r-- 1 root root 1.1K Nov  5 11:09 server-req.pem

Now you do have the client's and server’s certificates you need to go ahead with this setup.

Setting Up GTID Replication SSL Based

If you got new servers, maybe it’s just easy enough to say that, to configure replication SSL based, you need to have a user for each of the slaves/replicas you plan to have under your master server or as well, you can have a specialized user create for connecting to your master from an IP using a wildcard, such as 192.168.100.%. I will encourage you to have one user per slave/replica as it can enforce the security of your environment and avoid other issues like someone else on the same network trying to gain access on the master database. It’s OK that the replication user just has REPLICATION SLAVE and REPLICATION CLIENT privileges, but, you never know what is going to be attempted. By the way, following what should be done, you have the following:

  1. Move client certificates to all 3 servers, adding certificates at /etc/my.cnf.d/certs/ (you need to create this directory on all four servers);
  2. Add a file under the /etc/my.cnf.d names ssl.cnf as MariaDB will read that when it starts up mysqld;
  3. Create the users, one for each of the slaves, on master with the directive REQUIRE SSL;
  4. Configure replication on slaves/replicas with that user and using the required MASTER_SSL directives on the CHANGE MASTER TO command.

To move files around, I like to have a key based authentication configured to makes things easier as you don't need to digit passwords anymore after getting keys in place on all the servers. You can generate you a key on each of the servers, copy them all them all to the ~/.ssh/authorized_keys file on the central servers, which in my case is the MariaDB MaxScale server host and them, send the files to all the servers. One additional thing you need to pay attention, in this case, is that the authorized_keys file should have permission set as 0600 to make it work. So, this is a way to go, or, you can use your user's password as well, it's going to work. You can as well for sure streamline the process like below (it's a terrible behavior generate a key without a passphrase, so, consider a passphrase to your keys to make it safer):

#: generate a simple key, you can have a strong one
#: if you go create it on production
[root@maxscale ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
[...snip...]

Let's get key published on database servers:

#: adding the public key on the other hosts
[root@maxscale ~]# for i in {11..13}; do ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.50.$i; done
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.50.11's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '192.168.50.11'"
and check to make sure that only the key(s) you wanted were added.

/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.50.12's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '192.168.50.12'"
and check to make sure that only the key(s) you wanted were added.

/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.50.13's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '192.168.50.13'"
and check to make sure that only the key(s) you wanted were added.

#: testing if key based SSH is all set
[root@maxscale ~]# for i in {11..13}; do ssh 192.168.50.$i hostname; done
box01
box02
box03

Once SSH keys are in place, we can just move the certificates from your central host to the others; I use rsync for the below task, and as a hint, you will need to have it on all servers:

#: moving certificates for database hosts
[root@maxscale ~]# for i in {11..13}; do rsync -avrP -e ssh /etc/my.cnf.d/certs/* 192.168.50.$i:/etc/my.cnf.d/certs/; done
sending incremental file list
ca-cert.pem
  1261 100% 0.00kB/s 0:00:00 (xfer#1, to-check=7/8)
ca-key.pem
  1675 100% 1.60MB/s 0:00:00 (xfer#2, to-check=6/8)
client-cert.pem
  1135 100% 1.08MB/s 0:00:00 (xfer#3, to-check=5/8)
client-key.pem
  1675 100% 1.60MB/s 0:00:00 (xfer#4, to-check=4/8)
client-req.pem
  976 100% 953.12kB/s 0:00:00 (xfer#5, to-check=3/8)
server-cert.pem
  1135 100% 1.08MB/s 0:00:00 (xfer#6, to-check=2/8)
server-key.pem
  1704 100% 1.63MB/s 0:00:00 (xfer#7, to-check=1/8)
server-req.pem
  976 100% 953.12kB/s 0:00:00 (xfer#8, to-check=0/8)
 
sent 11046 bytes received 164 bytes 22420.00 bytes/sec
total size is 10537 speedup is 0.94
[...snip...]

Once certificates are located on all servers, next step is to add the ssl.cnf at /etc/my.cnf.d, as below:

#: add the below as a content of the file /etc/my.cnf.d/ssl.cnf
[root@box01 ~]# cat /etc/my.cnf.d/ssl.cnf
[client]
ssl
ssl-ca=/etc/my.cnf.d/certs/ca-cert.pem
ssl-cert=/etc/my.cnf.d/certs/client-cert.pem
ssl-key=/etc/my.cnf.d/certs/client-key.pem
[mysqld]
ssl
ssl-ca=/etc/my.cnf.d/certs/ca-cert.pem
ssl-cert=/etc/my.cnf.d/certs/server-cert.pem
ssl-key=/etc/my.cnf.d/certs/server-key.pem

You should restart your MariaDB Server after adding the certificates configuration, as if you don't, it's not going to be possible to connect to the database server with the users we created. In case something goes wrong with certificates, and you need to generate new ones, repeating the process aforementioned, you'll need to restart database servers as client certificates are loaded to the memory, and you can get an error like below if you have a certificates mismatch:

[root@box01 certs]# mysql
ERROR 2026 (HY000): SSL connection error: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Let’s now create a specific replication user for each of the servers we have on our replication topology currently:

box01 [(none)]> CREATE USER repl_ssl@'192.168.50.11' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.00 sec)

box01 [(none)]> GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO repl_ssl@'192.168.50.11' REQUIRE SSL;
Query OK, 0 rows affected (0.00 sec)

box01 [(none)]> CREATE USER repl_ssl@'192.168.50.12' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.00 sec)

box01 [(none)]> GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO repl_ssl@'192.168.50.12' REQUIRE SSL;
Query OK, 0 rows affected (0.00 sec)

box01 [(none)]> CREATE USER repl_ssl@'192.168.50.13' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.00 sec)

box01 [(none)]> GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO repl_ssl@'192.168.50.13' REQUIRE SSL;
Query OK, 0 rows affected (0.00 sec)

Above, we created one user per server, and I did that thinking at the moment that we eventually need to switch over the current master to one of the slaves, so, that way, the replication user don’t need to be of concern when dealing with an emergency or even when a planned failover is required. The next step should be thought in your case, and I’m going to simplify the case here and assume we’re working with a new environment, not in production yet. For changing your production environment to use SSL certificates, you need to spend more time on this, planning it well to avoid services disruptions. So, I’m going to grab the replication coordinates on the master, out of SHOW MASTER STATUS and then, issue the command CHANGE MASTER TO on slaves to get replication going. Here, I assumed you moved all the certs to all database servers, and they are living at /etc/my.cnf.d/certs/.

#: getting the current master status
box01 [(none)]> show master status\G
*************************** 1. row ***************************
            File: box01-bin.000024
        Position: 877
    Binlog_Do_DB:
Binlog_Ignore_DB:
1 row in set (0.00 sec)

#: the CHANGE MASTER TO command should be something like the below
box02 [(none)]> CHANGE MASTER TO MASTER_HOST='192.168.50.11',
  -> MASTER_USER='repl_ssl',
  -> MASTER_PASSWORD='123456',
  -> MASTER_LOG_FILE='box01-bin.000024',
  -> MASTER_LOG_POS=877,
  -> MASTER_SSL=1,
  -> MASTER_SSL_CA='/etc/my.cnf.d/certs/ca-cert.pem',
  -> MASTER_SSL_CERT='/etc/my.cnf.d/certs/client-cert.pem',
  -> MASTER_SSL_KEY='/etc/my.cnf.d/certs/client-key.pem';
Query OK, 0 rows affected (0.05 sec)

box02 [(none)]> start slave;
Query OK, 0 rows affected (0.00 sec)

box02 [(none)]> show slave status\G
*************************** 1. row ***************************
  Slave_IO_State: Waiting for master to send event
  Master_Host: 192.168.50.11
  Master_User: repl_ssl
  Master_Port: 3306
  Connect_Retry: 3
  Master_Log_File: box01-bin.000028
  Read_Master_Log_Pos: 794
  Relay_Log_File: box02-relay-bin.000006
  Relay_Log_Pos: 1082
  Relay_Master_Log_File: box01-bin.000028
  Slave_IO_Running: Yes
  Slave_SQL_Running: Yes
  [...snip...]
  Master_SSL_Allowed: Yes
  Master_SSL_CA_File: /etc/my.cnf.d/certs/ca-cert.pem
  Master_SSL_CA_Path:
  Master_SSL_Cert: /etc/my.cnf.d/certs/client-cert.pem
  Master_SSL_Cipher:
  Master_SSL_Key: /etc/my.cnf.d/certs/client-key.pem
  Seconds_Behind_Master: 0
  Last_IO_Errno: 0
  Last_IO_Error:
  Last_SQL_Errno: 0
  Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
  Master_Server_Id: 1
  Master_SSL_Crl: /etc/my.cnf.d/certs/ca-cert.pem
  Master_SSL_Crlpath:
  Using_Gtid: No
  Gtid_IO_Pos:
  Replicate_Do_Domain_Ids:
  Replicate_Ignore_Domain_Ids:
  Parallel_Mode: conservative
1 row in set (0.00 sec)

You can use GTIDs as well, and then, you CHANGE MASTER TO command will be something like:

box03 [(none)]> CHANGE MASTER TO MASTER_HOST='192.168.50.11',
  -> MASTER_USER='repl_ssl',
  -> MASTER_PASSWORD='123456',
  -> MASTER_USE_GTID=SLAVE_POS,
  -> MASTER_SSL=1,
  -> MASTER_SSL_CA='/etc/my.cnf.d/certs/ca-cert.pem',
  -> MASTER_SSL_CERT='/etc/my.cnf.d/certs/client-cert.pem',
  -> MASTER_SSL_KEY='/etc/my.cnf.d/certs/client-key.pem';
Query OK, 0 rows affected (0.05 sec)

box03 [(none)]> start slave;
Query OK, 0 rows affected (0.04 sec)

box03 [(none)]> show slave status\G
*************************** 1. row ***************************
  Slave_IO_State: Waiting for master to send event
  Master_Host: 192.168.50.11
  Master_User: repl_ssl
  Master_Port: 3306
  Connect_Retry: 3
  Master_Log_File: box01-bin.000028
  Read_Master_Log_Pos: 794
  Relay_Log_File: box03-relay-bin.000002
  Relay_Log_Pos: 654
  Relay_Master_Log_File: box01-bin.000028
  Slave_IO_Running: Yes
  Slave_SQL_Running: Yes
  [...snip...]
  Master_SSL_Allowed: Yes
  Master_SSL_CA_File: /etc/my.cnf.d/certs/ca-cert.pem
  Master_SSL_CA_Path:
  Master_SSL_Cert: /etc/my.cnf.d/certs/client-cert.pem
  Master_SSL_Cipher:
  Master_SSL_Key: /etc/my.cnf.d/certs/client-key.pem
  Seconds_Behind_Master: 0
  Last_IO_Errno: 0
  Last_IO_Error:
  Last_SQL_Errno: 0
  Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
  Master_Server_Id: 1
  Master_SSL_Crl: /etc/my.cnf.d/certs/ca-cert.pem
  Master_SSL_Crlpath:
  Using_Gtid: Slave_Pos
  Gtid_IO_Pos: 0-1-911075
  Replicate_Do_Domain_Ids:
  Replicate_Ignore_Domain_Ids:
  Parallel_Mode: conservative
1 row in set (0.00 sec)

One of the things you can check at the end to make sure replication is all set is of course if error log gives you a clear view of everything that was set up until now and if you added the variable report_host, you could see the result of SHOW SLAVE HOSTS on the master like below:

box01 [(none)]> show slave hosts\G 
*************************** 1. row ***************************
Server_id: 3
  Host: box03.wb.com
  Port: 3306
Master_id: 1

*************************** 2. row ***************************
Server_id: 2
  Host: box02.wb.com
  Port: 3306
Master_id: 1
2 rows in set (0.00 sec)

Unfortunately, the @@report_host is not a dynamic system variable, and you need to add it to the MariaDB Server configuration file and restart mysqld to make it assume the new value.  It's passed to the master when the slave/replica's IO_THREAD establish the connection with the master (handshake process).   Setting Up MariaDB MaxScale and the ReadWriteSplit SSL Based   Until here, we went through the details of each of the configurations from the certificates generation, replication configuration, and setup. Now, we need to go over the MaxScale installation; the commands required to dynamically create the monitor, a service, a listener and add servers to have at the end the configurations for the ReadWriteSplit router to handle reads and writes for the master and slaves. The steps here will be:

  1. Setup MariaDB MaxScale;
  2. Put together a basic configuration for MariaDB MaxScale and start it;
  3. Create a user for the MaxScale's Service and another one for the Monitor on backends with the REQUIRE SSL;
  4. Add SSL certificates to the server's and listener definitions files;
  5. Run commands that will create a monitor, a listener, a service; we will then create the servers and add them to the monitor;
  6. Create a user for the application on backends.

To setup MariaDB MaxScale (when writing this blog, it was at its 2.1.10 version), run the below knowing that the MariaDB Official repository was set up at the very beginning of this exercise:

#: setting up MariaDB Maxscale
[root@maxscale ~]# yum install maxscale -y

Loaded plugins: fastestmirror

Loading mirror speeds from cached hostfile
 * base: mirror.nbtelecom.com.br
 * epel: mirror.cedia.org.ec
 * extras: mirror.nbtelecom.com.br
 * updates: mirror.nbtelecom.com.br

Resolving Dependencies
--> Running transaction check
---> Package maxscale.x86_64 0:2.1.10-1 will be updated
---> Package maxscale.x86_64 0:2.1.11-1 will be an update
--> Finished Dependency Resolution
[...snip...]

[root@maxscale maxscale.cnf.d]# maxscale --version
MaxScale 2.1.11

You will notice that the password for the maxuser_ssl and maxmon_ssl users is a kind of hash. It was generated using maxkeys to avoid clean text, as you can see below. You will be required to configure yours instead of using the below one.

#: create the secrets file, by default at /var/lib/maxscale 
[root@maxscale ~]# maxkeys
Generating .secrets file in /var/lib/maxscale.

#: the password configured on database servers, but encrypted for maxscale configs
[root@maxscale ~]# maxpasswd /var/lib/maxscale/ 123456
AF76BE841B5B4692D820A49298C00272

#: change the file /var/lib/maxscale/.secrets ownership
[root@maxscale ~]# chown maxscale:maxscale /var/lib/maxscale/.secrets

Let's now put together a basic configuration to start MariaDB Maxscale. Add the below configurations to MariaDB MaxScale's configuration file so you can start MaxScale:

[root@maxscale ~]# cat /etc/maxscale.cnf
[maxscale]
threads=auto
log_info=true

[rwsplit-service]
type=service
router=readwritesplit
user=maxuser_ssl
passwd=AF76BE841B5B4692D820A49298C00272

[CLI]
type=service
router=cli

[CLI Listener]
type=listener
service=CLI
protocol=maxscaled
socket=default

Before starting MariaDB MaxScale, adding a listener to the pre-defined service, a monitor and creating and adding our servers on which we set up the replication previously, we need to create users, for the service, the monitor and for the application that will connect to the backend servers through MaxScale. Below users should be created on master and then, replicate for the replicas:

#: maxscale's mysqlmon user
sudo mysql -e "grant all on *.* to maxmon_ssl@'192.168.50.100' identified by '123456' require ssl" -vvv

#: maxscale's service user
sudo mysql -e "grant all on *.* to maxuser_ssl@'192.168.50.100' identified by '123456' require ssl" -vvv

#: application user
sudo mysql -e "grant select,insert,delete,update on *.* to appuser_ssl@'192.168.%' identified by '123456' require ssl;" -vvv

Now we can start MariaDB Maxscale using the basic configuration file we just created; I created mine at /root/maxscale_configs. So, I can start Maxscale doing like below and checking the log file at /var/log/maxscale/maxscale.log:

[root@maxscale maxscale.cnf.d]# [root@maxscale certs]# systemctl start maxscale
[root@maxscale certs]# systemctl status maxscale
● maxscale.service - MariaDB MaxScale Database Proxy
   Loaded: loaded (/usr/lib/systemd/system/maxscale.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2017-12-15 13:21:57 GMT; 5s ago
  Process: 13138 ExecStart=/usr/bin/maxscale (code=exited, status=0/SUCCESS)
  Process: 13135 ExecStartPre=/usr/bin/install -d /var/run/maxscale -o maxscale -g maxscale (code=exited, status=0/SUCCESS)
 Main PID: 13140 (maxscale)
   CGroup: /system.slice/maxscale.service
           └─13140 /usr/bin/maxscale

Dec 15 13:21:57 maxscale maxscale[13140]: Loaded module readwritesplit: V1.1.0 from /usr/lib64/maxscale/libreadwritesplit.so
Dec 15 13:21:57 maxscale maxscale[13140]: Loaded module maxscaled: V2.0.0 from /usr/lib64/maxscale/libmaxscaled.so
Dec 15 13:21:57 maxscale maxscale[13140]: Loaded module MaxAdminAuth: V2.1.0 from /usr/lib64/maxscale/libMaxAdminAuth.so
Dec 15 13:21:57 maxscale maxscale[13140]: No query classifier specified, using default 'qc_sqlite'.
Dec 15 13:21:57 maxscale maxscale[13140]: Loaded module qc_sqlite: V1.0.0 from /usr/lib64/maxscale/libqc_sqlite.so
Dec 15 13:21:57 maxscale maxscale[13140]: Service 'rwsplit-service' has no listeners defined.
Dec 15 13:21:57 maxscale maxscale[13140]: Listening for connections at [/tmp/maxadmin.sock]:0 with protocol MaxScale Admin
Dec 15 13:21:57 maxscale maxscale[13140]: MaxScale started with 1 server threads.
Dec 15 13:21:57 maxscale maxscale[13140]: Started MaxScale log flusher.
Dec 15 13:21:57 maxscale systemd[1]: Started MariaDB MaxScale Database Proxy.

At this point, the maxscale does not have anything to report in but the service we configured on the basic configuration file. That is mandatory to start Maxscale and make it happy on the first basic initialization. The log events above can show you that maxscale was started with a service but not a listener, not a monitor and no servers. So, this is what we're going to create now, running the below commands while checking the Maxscale's log file (below were extracted from this blog from Marküs Mäkelä and adjusted/fixed on this JIRA):

#: let's create a monitor based on the "mysqlmon"
[root@maxscale maxscale_configs]# maxadmin create monitor cluster-monitor mysqlmon
Created monitor 'cluster-monitor'
 
#: log file will tell the below
2017-10-10 15:34:31   notice : (3) [mysqlmon] Initialise the MySQL Monitor module.

#: let's alter the monitor to add some options 
[root@maxscale maxscale_configs]# maxadmin alter monitor cluster-monitor user=maxuser_ssl password=AF76BE841B5B4692D820A49298C00272 monitor_interval=10000
 
#: log file will tell you about the last changes
2017-10-10 15:34:31   notice : (3) Loaded module mysqlmon: V1.5.0 from /usr/lib64/maxscale/libmysqlmon.so
2017-10-10 15:34:31   notice : (3) Created monitor 'cluster-monitor'
2017-10-10 15:35:03   notice : (4) Updated monitor 'cluster-monitor': type=monitor
2017-10-10 15:35:03   notice : (4) Updated monitor 'cluster-monitor': user=maxuser_ssl
2017-10-10 15:35:03   notice : (4) Updated monitor 'cluster-monitor': password=AF76BE841B5B4692D820A49298C00272
2017-10-10 15:35:03   notice : (4) Updated monitor 'cluster-monitor': monitor_interval=1000

#: let's restart the monitor to take changes in effect
[root@maxscale maxscale_configs]# maxadmin restart monitor cluster-monitor
2017-10-10 18:40:50   error  : [mysqlmon] No Master can be determined

#: let's list existing monitors
[root@maxscale maxscale_configs]# maxadmin list monitors
---------------------+---------------------
Monitor              | Status
---------------------+---------------------
cluster-monitor      | Running
---------------------+---------------------

#: let’s create the listener, adding the client certificates for the connections
[root@maxscale maxscale.cnf.d]# maxadmin create listener rwsplit-service rwsplit-listener 0.0.0.0 4006 default default default /etc/my.cnf.d/certs/client-key.pem /etc/my.cnf.d/certs/client-cert.pem /etc/my.cnf.d/certs/ca-cert.pem
Listener 'rwsplit-listener' created
 
#: this is what log events tells us
2017-11-22 23:26:18 notice : (5) Using encrypted passwords. Encryption key: '/var/lib/maxscale/.secrets'.
2017-11-22 23:26:18 notice : (5) [MySQLAuth] [rwsplit-service] No users were loaded but 'inject_service_user' is enabled. Enabling service credentials for authentication until database users have been successfully loaded.
2017-11-22 23:26:18 notice : (5) Listening for connections at [0.0.0.0]:4006 with protocol MySQL
2017-11-22 23:26:18 notice : (5) Created TLS encrypted listener 'rwsplit-listener' at 0.0.0.0:4006 for service 'rwsplit-service'

#: listing the existing listeners 
[root@maxscale maxscale.cnf.d]# maxadmin list listeners
Listeners.

-----------------+---------------------+--------------------+-----------------+-------+--------
Name             | Service Name        | Protocol Module    | Address         | Port  | State
-----------------+---------------------+--------------------+-----------------+-------+--------
rwsplit-listener | rwsplit-service     | MySQLClient        | 0.0.0.0         | 4006  | Running
CLI Listener     | CLI                 | maxscale           | default         | 0     | Running
-----------------+---------------------+--------------------+-----------------+-------+--------

Here is the point in which you need to create the servers and then, you need alter the server’s configurations to add the SSL certificates, let’s see:

#: creating the server prod_mariadb01 and alter its configurations to add SSL
[root@maxscale ~]# maxadmin create server prod_mariadb01 192.168.50.11 3306
Created server 'prod_mariadb01'

[root@maxscale ~]# maxadmin alter server prod_mariadb01 ssl=required ssl_key=/etc/my.cnf.d/certs/client-key.pem ssl_cert=/etc/my.cnf.d/certs/client-cert.pem ssl_ca_cert=/etc/my.cnf.d/certs/ca-cert.pem

#: creating the server prod_mariadb02 and alter its configurations to add SSL
[root@maxscale ~]# maxadmin create server prod_mariadb02 192.168.50.12 3306
Created server 'prod_mariadb02'

[root@maxscale ~]# maxadmin alter server prod_mariadb02 ssl=required ssl_key=/etc/my.cnf.d/certs/client-key.pem ssl_cert=/etc/my.cnf.d/certs/client-cert.pem ssl_ca_cert=/etc/my.cnf.d/certs/ca-cert.pem
 
#: creating the server prod_mariadb03 and alter its configurations to add SSL
[root@maxscale ~]# maxadmin create server prod_mariadb03 192.168.50.13 3306
Created server 'prod_mariadb03'

[root@maxscale ~]# maxadmin alter server prod_mariadb03 ssl=required ssl_key=/etc/my.cnf.d/certs/client-key.pem ssl_cert=/etc/my.cnf.d/certs/client-cert.pem ssl_ca_cert=/etc/my.cnf.d/certs/ca-cert.pem

#: maxscale logs should be like
2017-12-02 18:56:28   notice : (19) Loaded module MySQLBackend: V2.0.0 from /usr/lib64/maxscale/libMySQLBackend.so
2017-12-02 18:56:28   notice : (19) Loaded module MySQLBackendAuth: V1.0.0 from /usr/lib64/maxscale/libMySQLBackendAuth.so
2017-12-02 18:56:28   notice : (19) Created server 'prod_mariadb01' at 192.168.50.11:3306
2017-12-02 18:57:57   notice : (20) Enabled SSL for server 'prod_mariadb01'
2017-12-02 19:00:42   notice : (22) Created server 'prod_mariadb02' at 192.168.50.12:3306
2017-12-02 19:00:49   notice : (23) Enabled SSL for server 'prod_mariadb02'
2017-12-02 19:00:58   notice : (24) Created server 'prod_mariadb03' at 192.168.50.13:3306
2017-12-02 19:01:04   notice : (25) Enabled SSL for server 'prod_mariadb03'

It’s good to say that MySQLBackend and MySQLBackedAuth are default values for the server’s protocol and the authenticator module respectively and those values are assumed by default when it’s omitted when creating servers. At this point we can show servers to see the servers configured with the SSL certificates:

[root@maxscale ~]# maxadmin show servers | grep -i ssl
    SSL initialized:                     yes
    SSL method type:                     MAX
    SSL certificate verification depth:  9
    SSL certificate:                     /etc/my.cnf.d/certs/client-cert.pem
    SSL key:                             /etc/my.cnf.d/certs/client-key.pem
    SSL CA certificate:                  /etc/my.cnf.d/certs/ca-cert.pem
    SSL initialized:                     yes
    SSL method type:                     MAX
    SSL certificate verification depth:  9
    SSL certificate:                     /etc/my.cnf.d/certs/client-cert.pem
    SSL key:                             /etc/my.cnf.d/certs/client-key.pem
    SSL CA certificate:                  /etc/my.cnf.d/certs/ca-cert.pem
    SSL initialized:                     yes
    SSL method type:                     MAX
    SSL certificate verification depth:  9
    SSL certificate:                     /etc/my.cnf.d/certs/client-cert.pem
    SSL key:                             /etc/my.cnf.d/certs/client-key.pem
    SSL CA certificate:                  /etc/my.cnf.d/certs/ca-cert.pem

And then, we can list servers, and you will see that, it’s not yet recognized by MaxScale being neither master or slave:

[root@maxscale maxscale_configs]# maxadmin list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server             | Address         | Port  | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
prod_mysql03       | 192.168.50.13   |  3306 |           0 | Running
prod_mysql02       | 192.168.50.12   |  3306 |           0 | Running
prod_mysql01       | 192.168.50.11   |  3306 |           0 | Running
-------------------+-----------------+-------+-------------+--------------------

Next step is to add the created servers to the monitor and service; both created previously as well:

[root@maxscale maxscale_configs]# maxadmin add server prod_mariadb01 cluster-monitor rwsplit-service
Added server 'prod_mysql01' to 'cluster-monitor'
Added server 'prod_mysql01' to 'rwsplit-service’

[root@maxscale maxscale_configs]# maxadmin add server prod_mariadb02 cluster-monitor rwsplit-service
Added server 'prod_mysql02' to 'cluster-monitor'
Added server 'prod_mysql02' to 'rwsplit-service'

[root@maxscale maxscale_configs]# maxadmin add server prod_mariadb03 cluster-monitor rwsplit-service
Added server 'prod_mysql03' to 'cluster-monitor'
Added server 'prod_mysql03' to 'rwsplit-service’

#: logs
2017-10-10 18:45:45   notice : (16) Added server 'prod_mysql01' to monitor 'cluster-monitor'
2017-10-10 18:45:45   notice : (16) Added server 'prod_mysql01' to service 'rwsplit-service'
2017-10-10 18:45:45   notice : Server changed state: prod_mysql01[192.168.50.11:3306]: new_master. [Running] -> [Master, Running]
2017-10-10 18:45:45   notice : [mysqlmon] A Master Server is now available: 192.168.50.11:3306
2017-10-10 18:45:52   notice : (17) Added server 'prod_mysql02' to monitor 'cluster-monitor'
2017-10-10 18:45:52   notice : (17) Added server 'prod_mysql02' to service 'rwsplit-service'
2017-10-10 18:45:53   notice : Server changed state: prod_mysql01[192.168.50.11:3306]: lost_master. [Master, Running] -> [Running]
2017-10-10 18:45:53   error  : [mysqlmon] No Master can be determined
2017-10-10 18:45:56   notice : (18) Added server 'prod_mysql03' to monitor 'cluster-monitor'
2017-10-10 18:45:56   notice : (18) Added server 'prod_mysql03' to service 'rwsplit-service'
2017-10-10 18:45:56   notice : Server changed state: prod_mysql01[192.168.50.11:3306]: new_master. [Running] -> [Master, Running]
2017-10-10 18:45:56   notice : Server changed state: prod_mysql03[192.168.50.13:3306]: new_slave. [Running] -> [Slave, Running]
2017-10-10 18:45:56   notice : [mysqlmon] A Master Server is now available: 192.168.50.11:3306

You can see that, when adding servers to the service, which is the ReadWriteSplit, the current servers’ states and their roles pops up.

[root@maxscale ~]# maxadmin list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server             | Address         | Port  | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
prod_mariadb01     | 192.168.50.11   |  3306 |           0 | Master, Running
prod_mariadb02     | 192.168.50.12   |  3306 |           0 | Slave, Running
prod_mariadb03     | 192.168.50.13   |  3306 |           0 | Slave, Running
-------------------+-----------------+-------+-------------+--------------------

Yet, all the configurations are modular and you can find all files created based on the dynamic configurations we have done until now at /var/lib/maxscale/maxscale.cnf.d:

[root@maxscale maxscale.cnf.d]# ls -lh
total 24K
-rw-r--r-- 1 root root 251 Dec  2 19:10 cluster-monitor.cnf
-rw-r--r-- 1 root root 299 Dec  2 18:57 prod_mariadb01.cnf
-rw-r--r-- 1 root root 299 Dec  2 19:00 prod_mariadb02.cnf
-rw-r--r-- 1 root root 299 Dec  2 19:01 prod_mariadb03.cnf
-rw-r--r-- 1 root root 313 Nov 22 23:26 rwsplit-listener.cnf
-rw-r--r-- 1 root root  71 Dec  2 19:10 rwsplit-service.cnf

SSL configurations will be on rwsplit-listener.cnf and on servers’ files:

[root@maxscale maxscale.cnf.d]# cat rwsplit-listener.cnf
[rwsplit-listener]
type=listener
protocol=MySQLClient
service=rwsplit-service
address=0.0.0.0
port=4006
authenticator=MySQLAuth
ssl=required
ssl_cert=/etc/my.cnf.d/certs/client-cert.pem
ssl_key=/etc/my.cnf.d/certs/client-key.pem
ssl_ca_cert=/etc/my.cnf.d/certs/ca-cert.pem
ssl_cert_verify_depth=9
ssl_version=MAX
 
[root@maxscale maxscale.cnf.d]# cat prod_mariadb0*
[prod_mariadb01]
type=server
protocol=MySQLBackend
address=192.168.50.11
port=3306
authenticator=MySQLBackendAuth
ssl=required
ssl_cert=/etc/my.cnf.d/certs/client-cert.pem
ssl_key=/etc/my.cnf.d/certs/client-key.pem
ssl_ca_cert=/etc/my.cnf.d/certs/ca-cert.pem
ssl_cert_verify_depth=9
ssl_version=MAX
 
[prod_mariadb02]
type=server
protocol=MySQLBackend
address=192.168.50.12
port=3306
authenticator=MySQLBackendAuth
ssl=required
ssl_cert=/etc/my.cnf.d/certs/client-cert.pem
ssl_key=/etc/my.cnf.d/certs/client-key.pem
ssl_ca_cert=/etc/my.cnf.d/certs/ca-cert.pem
ssl_cert_verify_depth=9
ssl_version=MAX
 
[prod_mariadb03]
type=server
protocol=MySQLBackend
address=192.168.50.13
port=3306
authenticator=MySQLBackendAuth
ssl=required
ssl_cert=/etc/my.cnf.d/certs/client-cert.pem
ssl_key=/etc/my.cnf.d/certs/client-key.pem
ssl_ca_cert=/etc/my.cnf.d/certs/ca-cert.pem
ssl_cert_verify_depth=9
ssl_version=MAX

At this point, as everything is set up, you can test the access to your databases through MaxScale, using the appuser_ssl (If you haven’t created that user yet, create it now on the master and check authentication). You will notice the below event added to the MaxScale's log as of when you create new users as Maxscale will update its internal information about users on backends:

2017-12-03 00:15:17   notice : [MySQLAuth] [rwsplit-service] Loaded 15 MySQL users for listener rwsplit-listener.

If you have the user created, as we did create it before, place the below contents at the home directory of your user and test the access with the appuser_ssl user.

#: check if mysql client is present on Maxscale server
[root@maxscale ~]# which mysql
/bin/mysql

#: add the .my.cnf at your user's home directory
[root@maxscale ~]# cat .my.cnf
[client]
ssl
ssl-ca=/etc/my.cnf.d/certs/ca-cert.pem
ssl-cert=/etc/my.cnf.d/certs/client-cert.pem
ssl-key=/etc/my.cnf.d/certs/client-key.pem
[mysql]
ssl
ssl-ca=/etc/my.cnf.d/certs/ca-cert.pem
ssl-cert=/etc/my.cnf.d/certs/client-cert.pem
ssl-key=/etc/my.cnf.d/certs/client-key.pem

To execute the below test you will need to install MariaDB-client package on the MariaDB Maxscale server host.

[root@maxscale ~]# mysql -u appuser_ssl -p123456 -h 192.168.50.100 -P 4006 -e "select @@server_id\G"
*************************** 1. row ***************************
@@server_id: 2

[root@maxscale ~]# mysql -u appuser_ssl -p123456 -h 192.168.50.100 -P 4006 -e "select @@server_id\G"
*************************** 1. row ***************************
@@server_id: 3

Conclusion

It's a very dense reading, full of practices, bells and whittles, but, it's going to serve as a reference for you when implementing MariaDB MaxScale, with traffic going over SSL. This is not only about MaxScale, but, about having MariaDB Servers with data being replicated using SSL certificates as well. Remember that, as MaxScale Dynamic Commands made it possible to configure ReadWriteSplit with mysqlmon, it gives you the same as well to work with galeramon. The product is becoming more and more versatile and the main point to highlight is it's making the task to position a load balancer or an intelligent database proxy between backends and the clients an easy thing.

This blog post will explore the setup of an environment using self-signed OpenSSL certificates to make it safe enough to replicate data between the multiple backend database servers and also, we’ll show you how to setup the communication between MariaDB MaxScale and the backend.

Login or Register to post comments

by Wagner Bianchi at January 09, 2018 06:44 PM

Jean-Jerome Schmidt

How To Achieve PCI Compliance for MySQL & MariaDB with ClusterControl - The Webinar

Join Laurent Blume, Unix Systems Engineer & PCI Specialist and Vinay Joosery, CEO at Severalnines, as they discuss all there is to know about how to achieve PCI compliance for MySQL & MariaDB with ClusterControl in this new webinar on January 30th.

The Payment Card Industry Data Security Standard (PCI-DSS) is a set of technical and operational requirements defined by the PCI Security Standards Council (PCI SSC) to protect cardholder data. These standards apply to all entities that store, process or transmit cardholder data – with requirements for software developers and manufacturers of applications and devices used in those transactions.

PCI data that resides in a MySQL or MariaDB database must of course also adhere to these requirements, and database administrators must follow best practices to ensure the data is secured and compliant. The PCI standards are stringent and can easily require a spiraling amount of time spent on meeting their requirements. Database administrators can end up overwhelmed when using software that was not designed for compliance, often because it long predates PCI itself, as is the case for most database systems in use today.

That is why, as often as possible, reliable tools must be chosen to help with that compliance, easing out the crucial parts. Each time the compliance for one requirement can be shown to be implemented, working, and logged accordingly, time will be saved. If well-designed, it will only require regular software upgrades, a yearly review and a moderate amount of tweaking to follow the standard's evolution over time.

This new webinar focuses on PCI-DSS requirements for a MySQL or MariaDB database back-end managed by ClusterControl in order to help meet these requirements. It will provide a MySQL and MariaDB user focussed overview of what the PCI standards mean, how they impact database management and provide valuable tips and tricks on how to achieve PCI compliance for MySQL & MariaDB with ClusterControl.

Sign up here!

Date, Time & Registration

Europe/MEA/APAC

Tuesday, January 30th at 09:00 GMT / 10:00 CET (Germany, France, Sweden)

Register Now

North America/LatAm

Tuesday, January 30th at 09:00 PT (US) / 12:00 ET (US)

Register Now

Agenda

  • Introduction to the PCI-DSS standards
  • The impact of PCI on database management
  • Step by step review of the PCI requirements
  • How to meet the requirements for MySQL & MariaDB with ClusterControl
  • Conclusion
  • Q&A

Speakers

Laurent Blume, Unix Systems Engineer, PCI Specialist

Laurent’s career in IT started in 2000, his work since evolved from POS terminals for a jewelry store chain to infrastructure servers in a government aerospace R&D organization, even touching supercomputers. One constant throughout was the increasing need for security.

For the past 6 years, he has been in charge of first implementing, then keeping up with the PCI-DSS compliance of critical transnational payment authorization systems. Its implementation for databases has been an essential part of the task. For the last few years, it has expanded to the design and productization of MariaDB cluster backends for mobile contactless payments.

Vinay Joosery, CEO & Co-Founder, Severalnines

Vinay is a passionate advocate and builder of concepts and business around distributed database systems.

Prior to co-founding Severalnines, Vinay held the post of Vice-President EMEA at Pentaho Corporation - the Open Source BI leader. He has also held senior management roles at MySQL / Sun Microsystems / Oracle, where he headed the Global MySQL Telecoms Unit, and built the business around MySQL's High Availability and Clustering product lines. Prior to that, Vinay served as Director of Sales & Marketing at Ericsson Alzato, an Ericsson-owned venture focused on large scale real-time databases.

by jj at January 09, 2018 10:55 AM

January 04, 2018

MariaDB AB

MariaDB Server 10.2.12 now available

MariaDB Server 10.2.12 now available dbart Thu, 01/04/2018 - 12:56

The MariaDB project is pleased to announce the immediate availability of MariaDB Server 10.2.12. See the release notes and changelog for details and visit mariadb.com/downloads to download.

Download MariaDB Server 10.2.12

Release Notes Changelog What is MariaDB 10.2?

The MariaDB project is pleased to announce the immediate availability of MariaDB Server 10.2.12. See the release notes and changelog for details.

Login or Register to post comments

by dbart at January 04, 2018 05:56 PM

MariaDB Foundation

MariaDB 10.2.12 now available

The MariaDB project is pleased to announce the availability of MariaDB 10.2.12, the next stable release in the 10.2 series. See the release notes and changelogs for details. Download MariaDB 10.2.12 Release Notes Changelog What is MariaDB 10.2? MariaDB APT and YUM Repository Configuration Generator Thanks, and enjoy MariaDB!

The post MariaDB 10.2.12 now available appeared first on MariaDB.org.

by Ian Gilfillan at January 04, 2018 05:17 PM

Jean-Jerome Schmidt

Announcing ClusterControl 1.5.1 - Featuring Backup Encryption for MySQL, MongoDB & PostgreSQL

What better way to start a new year than with a new product release?

Today we are excited to announce the 1.5.1 release of ClusterControl - the all-inclusive database management system that lets you easily deploy, monitor, manage and scale highly available open source databases - and load balancers - in any environment: on-premise or in the cloud.

ClusterControl 1.5.1 features encryption of backups for MySQL, MongoDB and PostgreSQL, a new topology viewer, support for MongoDB 3.4, several user experience improvements and more!

Feature Highlights

Full Backup and Restore Encryption for these supported backup methods

  • mysqldump, xtrabackup (MySQL)
  • pg_dump, pg_basebackup (PostgreSQL)
  • mongodump (MongoDB)

New Topology View (BETA) shows your replication topology (including load balancers) for your entire cluster to help you visualize your setup.

  • MySQL Replication Topology
  • MySQL Galera Topology

Improved MongoDB Support

  • Support for MongoDB v3.4
  • Fix to add back restore from backup
  • Multiple NICs support. Management/public IPs for monitoring connections and data/private IPs for replication traffic

Misc

Improved user experience featuring a new left-side navigation that includes:

  • Global settings breakout to make it easier to find settings related to a specific feature
  • Quick node actions that allow you to quickly perform actions on your node
ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

View Release Details and Resources

Improving Database Security: Backup & Restore Encryption

ClusterControl 1.5 introduces another step to ensuring your databases are kept secure and protected.

Backup & restore encryption means that backups are encrypted at rest using AES-256 CBC algorithm. An auto generated key will be stored in the cluster's configuration file under /etc/cmon.d. The backup files are transferred in encrypted format. Users can now secure their backups for offsite or cloud storage with the flip of a checkbox. This feature is available for select backup methods for MySQL, MongoDB & PostgreSQL.

New Topology View (beta)

This exciting new feature provides an “overhead” topology view of your entire cluster, including load balancers. While in beta, this feature currently supports MySQL Replication and Galera topologies. With this new feature, you can drag and drop to perform node actions. For example, you can drag a replication slave on top of a master node - which will prompt you to either rebuild the slave or change the replication master.

Improved User Experience

The new Left Side Navigation and the new quick actions and settings that accompany it mark the first major redesign to the ClusterControl interface in some time. ClusterControl offers a vast array of functionality, so much so that it can sometimes be overwhelming to the novice. This addition of the new navigation allows the user quick access to what they need on a regular basis and the new node quick actions lets users quickly run common commands and requests right from the navigation.

Download the new ClusterControl or request a demo.

by jj at January 04, 2018 01:41 PM

January 02, 2018

Valeriy Kravchuk

Fun with Bugs #59 - On MySQL Bug Reports I am Subscribed to, Part II

New Year (that starts on Monday!) gives a good opportunity to change something in our lives, start doing something new, better or different. Let's assume I failed with all these so far, as I am again posting about MySQL bugs here.

Since my previous post on this topic I've subscribed to 15 more MySQL bugs, and being on a combination of public holidays and vacation now gives me a good opportunity to review these bug reports.

Here they are, starting from the most recent:
  • Bug #89065 - "sync_binlog=1 on a busy server and slow binary log filesystem stalls slaves". I do not remember seeing multiple threads in "Finished reading one binlog; switching to next binlog" state, but it would be interesting to see this bug report processed properly.
  • Bug #89051 - "EXPLAIN output is different for same content depending when index was added". The way optimizer decides on "range" vs "ref" access is always interesting. Here, based on a recent comment by Øystein Grøvlen, the bug is actually that "Cost values are not correct when optimizer switch from ref-access to range-access in order to use more key parts".
  • Bug #88914 - "Potential null pointer dereference at pointer node->undo_recs (row0purge.cc)". It's funny to see many bugs becoming private as "security" ones and, at the same time, this bug, where reporter suspects it is exploitable, being "Open" and ignored for more than two weeks...
  • Bug #88891 - "Filtered replication leaves GTID holes with create database if not exists". I can not even explain how much I "like" all kinds of GTIDs I have to deal with, especially such a long lists of GTIDs that may be created in cases described in this report.
  • Bug #88863 - "COUNT(*) can sometimes return bogus value". Now, this is a really funny bug! It must be some race condition, and I'd really prefer to see this bug fixed soon.
  • Bug #88844 - "MySQL crash with InnoDB assertion failure in file pars0pars.cc". Nice crash (that I've never seen before) quickly reproduced by Shane Bester.
  • Bug #88834 - "Uneven slowdown on systems with many users". What can be better to speed up connection than checking the list of users one by one, especially when there are thousands of users?
  • Bug #88827 - "innodb uses too much space in the PK for concurrent inserts into the same table". As Mark Callaghan put it:
    "I am not sure my reproduction details will ever satisfy Sinisa but I don't mind if you don't fix this because I care more about MyRocks today and this bug makes MyRocks look better."
    We (Facebook's MySQL, MariaDB and Percona server users) do have MyRocks, but why poor Oracle MySQL users should suffer? Let's hope Sinisa Milivojevic will process the bug fast, with all the details clarified there :)
  • Bug #88791 - "Binary log for generated column contains new value as WHERE clause, not old value". Generated columns and binary logging, what could went wrong?
  • Bug #88764 - ""ALTER TABLE MODIFY..." takes time even if leaving table as is". Any simple test cases they come to my mind do NOT let to reproduce this problem, but I feel some potential as soon as more exotic cases like partitioning or data directory settings are considered. Let's wait for bug reporter to clarify.
  • Bug #88720 - "Inconsistent and unsafe FLUSH behavior in terms of replication". Nice summary of problems from Przemyslaw Malkowski. One more reason for me to hate GTIDs, honestly.
  • Bug #88694 - "MySQL accepts wildcard for database name for table level grant but won't use it". One more problem with privileges reported by Daniël van Eeden.
  • Bug #88674  - "Regression CREATE TBL from 5.7.17 to 20 (part #2: innodb_file_per_table = OFF)." and Bug #88673 - "Regression CREATE TBL from 5.7.17 to 20 (part #1: innodb_file_per_table = ON)." - these two bugs were reported by Jean-François Gagné and clearly show some things that are done wrong by Oracle when fixing (private, "security") bugs...
  • Bug #88671 - "ALL + BNL chosen over REF in query plan with a simple SELECT + JOIN". In this case optimizer (probably) does not take costs into account properly when choosing block nested loop join vs usual "ref" index access. Maybe just a matter of missing/wrong statistics also. It would be interesting to find out eventually.
  • Bug #88666 - "I_S FILES : all rows are displayed whatever the user privileges". Yet another bug report from Jocelyn Fournier. I am actually surprised with a number of bugs related to privileges that I notice recently.
  • Bug #88633 - "Auto_increment value on a table is less than max(id) can happen". It seems only MySQL 5.7.x is affected, but not 5.6.x.
  • Bug #88623 - "Modifying virtually generated column is not online". May be by design, but still surprising.
  • Bug #88534 - "XA may lost prepared transaction and cause different between master and slave". XA transactions with MySQL is still a sure way to disaster, I think. See Bug #87526 also that should appear in the next part of this series...
Stay tuned to more posts about MySQL bugs from me in the New Year of 2018!

by Valeriy Kravchuk (noreply@blogger.com) at January 02, 2018 08:15 PM

Colin Charles

Premier Open Source Database Conference Call for Papers closing January 12 2018

The call for papers for Percona Live Santa Clara 2018 was extended till January 12 2018. This means you still have time to get a submission in.

Topics of interest: MySQL, MongoDB, PostgreSQL & other open source databases. Don’t forget all the upcoming databases too (there’s a long list at db-engines).

I think to be fair, in the catch all “other”, we should also be thinking a lot about things like containerisation (Docker), Kubernetes, Mesosphere, the cloud (Amazon AWS RDS, Microsoft Azure, Google Cloud SQL, etc.), analytics (ClickHouse, MariaDB ColumnStore), and a lot more. Basically anything that would benefit an audience of database geeks whom are looking at it from all aspects.

That’s not to say case studies shouldn’t be considered. People always love to hear about stories from the trenches. This is your chance to talk about just that.

by Colin Charles at January 02, 2018 10:21 AM

Federico Razzoli

My 2018 Databases Wishlist

Well, the most important wishes I have for 2018 are a bit out of topic for this blog: forms of organisation without a formal authority, schools not teaching religions, and so on. But in this post, I will write about databases… as usual.

So, here is my whishlist, for what it matters.

More research on Learned Indexes

If you don’t know what I’m talking about, see this paper. Having a data structure faster than B-Trees is exciting. But of course I’d like to see also considerations on write performance.

Progress on using ML for database tuning

See this article. I don’t think that Machine Learning will ever be able to replace (good) DBAs, but having a tool which suggests tuning based on real workload sounds great. It can be a valid help for DBAs.

More research on stored functions transformation

Stored functions are useful but slow. But see this paper. It seems it is possible to transform imperative programs to queries, improving the complexity by some orders of magnitude.

On a side note, MariaDB implemented a lot of syntax from Oracle for stored procedures. While this sounds like a precise commercial strategy, the technical improvement on this area is great. Still, what I’d like to see is better performance, as well as support for external languages.

Galera 4

Let me be clear, I didn’t read any announcement that Galera 4 will be released this year. But they announced exciting news over time, and still the new version isn’t here. At some point, it should be released (hopefully).

Transactional DDL in the MySQL ecosystem

MySQL 8.0 has support for atomic DDL statements. They did it in a good way: it’s engine independent and, while it uses InnoDB information_schema tables, any engine is free to add support for this feature. They stated that this is the basis for transactional DDL, but we are not yet there. MariaDB has a task for transactional DDL.

EDIT: Thanks to Valerii Kravchuk for pointing me MDEV-11424 – Instant ALTER TABLE of failure-free record format changes. It is clearly worth adding it to my wishlist: please Maria, get it done!

Engines, engines, engines

RocksDB is great, please consolidate it. TokuDB can improve in many ways, please don’t stop investing on it. Next version of SPIDER will be in MariaDB 10.3, I hope that the development will be a bit more active in the future.

Don’t kill MyISAM. It is still useful in some cases. For Catawiki use cases, I find it better than InnoDB for temporary tables. Also JFG has a great use case example.

More progress on Tarantool and CockroachDB

Tarantool is a great database, originally NoSQL. It is extremely scriptable (actually it can be seen as a Lua application server) and its modules allow to read and write data from a wide variety of data sources, including MySQL replication. Recently, SQL support has been added.

CockroachDB is an open source RDBMS design to scale geographically. It uses distributed transaction. It also allows to tune the redundancy of data at table level and define replication zones.

Great conferences

I will be both at M18 (I’m not sponsored by my company, but I chosen to go anyway) and Percona Live. At M18 I will give a talk titled Somewhere between schema and schemaless. Of course I also submitted proposal for Percona Live, let’s see if they get accepted.

by Federico at January 02, 2018 12:36 AM

December 28, 2017

Jean-Jerome Schmidt

Our Most Popular Database Blog Posts in 2017

As we wrap up our last blog of 2017 we wanted to reflect on what content we have been creating that’s been resonating and generating the most interest with our readers. We will continue to deliver the best technical content we can for MySQL, Galera Cluster, PostgreSQL, MariaDB, and MongoDB in 2018.

Here is some of our most popular content from 2017…

Top Database Blogs for 2017

ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

Top Blogs by Technology

While MySQL and MySQL Galera Cluster dominate our most popular content we blog about a lot of different technologies and methodologies on the Severalnines blog. Here are some of the most popular blogs in 2017 for non-MySQL topics.

If there are some blog topics you would like us to cover in 2018 please list them in the comments below.

by fwlymburner at December 28, 2017 10:50 AM

MariaDB AB

MariaDB Holiday Reflections and Gratitude

MariaDB Holiday Reflections and Gratitude MariaDB Team Wed, 12/27/2017 - 19:44

As we wrap up 2017, we want to take a moment to reflect on the past year. We’ve seen tremendous support from our community, customers and partners – and for that, we are grateful. Our customers across the global have shared heartwarming stories of their business success and we appreciate the opportunity to be your valued partner.

For us, 2017 was a milestone year. Our progress this year reflects a global theme that brings us forward to new opportunities.

  • Solutions to global problems: To make it easier for our customers to find solutions specific to their types of workloads, we introduced two complete solutions – MariaDB TX for transactional workloads and MariaDB AX for analytical workloads. The solutions are comprised of our technology products, as well as tools, services and support for an optimal production deployment environment. This year was also highlighted by the GA releases of MariaDB Server 10.2, MariaDB MaxScale 2.1, MariaDB ColumnStore 1.1.

  • Global event: In April, we held our first annual MariaDB global user conference M|17. Customer from as far away as Asia and Europe traveled to NYC to share success stories, best practices and technical deep dives from the MariaDB team.

  • Growing reach worldwide: This year, Debian followed the trend of Linux distributions replacing MySQL with MariaDB as the default. MariaDB reaches more than 60 million developers worldwide through its inclusion in every major Linux distribution, as well as a growing presence in the world’s major cloud providers.

  • Global team to support global customers: We’ve expanded our diverse team with employees in 20+ countries, plus Jon Bakke joined the leadership ranks as Chief Revenue Officer.

  • Investment from all corners of the world: Additional investment from Alibaba, the European Investment Bank and existing investors continue to fund global growth and product innovation.

We are excited to keep up the momentum in 2018, starting with our second annual MariaDB M|18 user conference in February. We look forward to seeing you in NYC!

Wishing you a happy new year from the team at MariaDB!
 

Happy New Year | Onnellista uutta vuotta | Feliz año nuevo 

Bonne année | Gott Nytt År | Felice Anno Nuovo

Frohes neues Jahr | С Новым годом | 新年快乐 | 良いお年を | 새해 복 많이 받으세요

 

As we wrap up 2017, we want to take a moment to reflect on the past year. We’ve seen tremendous support from our community, customers and partners – and for that, we are grateful. Our customers across the global have shared heartwarming stories of their business success and we appreciate the opportunity to be your valued partner. For us, 2017 was a milestone year. Our progress this year reflects a global theme that brings us forward to new opportunities.

Login or Register to post comments

by MariaDB Team at December 28, 2017 12:44 AM

December 23, 2017

MariaDB Foundation

MariaDB 10.3.3 and MariaDB Connector/J 2.2.1 and 1.7.1 now available

The MariaDB project is pleased to announce the availability of MariaDB 10.3.3, the first beta release in the MariaDB 10.3 series, as well as MariaDB Connector/J 2.2.1, the latest stable release in the MariaDB Connector/J 2.2 series, and MariaDB Connector/J 1.7.1, the latest stable release in the MariaDB Connector/J 1.7 series. See the release notes […]

The post MariaDB 10.3.3 and MariaDB Connector/J 2.2.1 and 1.7.1 now available appeared first on MariaDB.org.

by Ian Gilfillan at December 23, 2017 03:28 PM

MariaDB AB

Announcing MariaDB Server 10.3 Beta

Announcing MariaDB Server 10.3 Beta RalfGebhardt Sat, 12/23/2017 - 10:05

We are happy to announce the first beta release of MariaDB Server 10.3, the fastest growing open source relational database. Beta is an important time in our release and we encourage you to download this release today! Please note that we do not recommend running beta releases in production.

MariaDB Server 10.3 is the next evolution, after MariaDB Server 10.2 with enhancements like Window Functions, Common Table Expressions, JSON functions and CHECK constraints. For MariaDB Server 10.3 a lot of effort has been spent on database compatibility enhancements, especially for stored routines. This will allow easier migration of stored functions and better usability of stored functions in general.

Now, with MariaDB Server 10.3.3 beta, new significant features and enhancements are available for our users and customers, including:

  • Database Compatibility Enhancements
    • PL/SQL Compatibility for MariaDB Stored Functions: The server now understands a subset of Oracle's PL/SQL language instead of the traditional MariaDB syntax for stored routines
    • New option for CURSOR in stored routines: A CURSOR can now have parameters used by the associated query
    • New data types for stored routines: ROW data type, TYPE OF and ROW TYPE OF anchored data types
    • Generation of unique primary keys by SEQUENCES: As an alternative to AUTO INCREMENT It is now possible to define names sequence objects to create a sequence of numeric values
    • Operations over result sets with INTERSECT and EXCEPT: In addition to the already existing UNION an intersection and subtraction of result sets is now possible
    • Define Columns to be invisible: Columns now can be defined to be invisible. There exist 3 levels of invisibility, user defined, system level and completely invisible
    • Window Function Enhancement: percentile and median window functions have been added
  • User Flexibility
    • User Defined Aggregate Functions: In addition to creating SQL functions it is now also possible to create aggregate functions
    • Lifted limitations for updates and deletes: A DELETE statement can now delete from a table used in the WHERE clause. UPDATE can be the same for source and target
  • Performance/Storage Enhancements
  • Storage Engine Enhancements
    • Spider Storage Engine: The partitioning storage engine has been updated to the newest release of the Spider Storage engine to support new Spider features including direct join support, direct update and delete, direct aggregates
    • Proxy Layer Support for MariaDB Server: Client / Server authentication via a Proxy like MariaDB MaxScale using a Server Proxy Protocol Support

Try out MariaDB Server Beta software and share your feedback

Download MariaDB Server 10.3.3 Beta

Release Notes Changelog What is MariaDB Server 10.3?

We are happy to announce the first beta release of MariaDB Server 10.3, the fastest growing open source relational database.

MariaDB Server 10.3 is the next evolution, after MariaDB Server 10.2 with enhancements like Window Functions, Common Table Expressions, JSON functions and CHECK constraints.

For MariaDB Server 10.3 a lot of effort has been spent on database compatibility enhancements, especially for stored routines.

Login or Register to post comments

by RalfGebhardt at December 23, 2017 03:05 PM

December 22, 2017

MariaDB Foundation

MariaDB 10.1.30 and MariaDB Connector/C 2.3.4 now available

The MariaDB project is pleased to announce the availability of MariaDB 10.1.30, the latest stable release in the MariaDB 10.1 series, as well as MariaDB Connector/C 2.3.4, the latest stable release in the MariaDB Connector/C 2.x series. See the release notes and changelogs for details. Download MariaDB 10.1.30 Release Notes Changelog What is MariaDB 10.1? […]

The post MariaDB 10.1.30 and MariaDB Connector/C 2.3.4 now available appeared first on MariaDB.org.

by Ian Gilfillan at December 22, 2017 11:29 PM

Peter Zaitsev

MongoDB 3.6 Sorting Changes and Useful New Features

MongoDB 3.6 sorting

MongoDB 3.6 sortingIn this blog, we will cover new MongoDB 3.6 sorting change and other useful features.

The new MongoDB 3.6 GA release can help you build applications and run them on MongoDB, and we will cover areas where there are changes to server behavior that may affect your application usage of MongoDB.

Sorting Changes

The most significant behavior change from an application perspective is sorting on arrays in find and aggregate queries. It is important to review this new behavior in a test environment when upgrading to MongoDB 3.6, especially if the exact order of sorted arrays in find results is crucial.

In MongoDB 3.6 sorting, a sort field containing an array is ordered with the lowest-valued element of the array first for ascending sorts and the highest-valued element of the array first for descending sorts. Before 3.6, it used the lowest-valued array element that matches the query.

Example: a collection has the following documents:

{ _id: 0, a: [-3, -2, 2, 3] }
{ _id: 1, a: [ 5, -4 ] }

And we perform this sort operation:

db.test.find({a: {$gte: 0}}).sort({a: 1});

In MongoDB 3.6 sorting, the sort operation no longer takes into account the query predicate when determining its sort key. The operation returns the documents in the following order:

{ "_id" : 1, "a" : [ 5, -4 ] }
{ "_id" : 0, "a" : [ -3, -2, 2, 3 ] }

Previous to 3.6 the result would be:

{ _id: 0, a: [-3, -2, 2, 3] }
{ _id: 1, a: [ 5, -4 ] }

More on this change here: https://docs.mongodb.com/manual/release-notes/3.6-compatibility/#array-sort-behavior and https://docs.mongodb.com/manual/release-notes/3.6-compatibility/#find-method-sorting.

Wire Protocol Compression

In MongoDB 3.4, wire protocol compression was added to the server and mongo shell using the snappy algorithm. However, this was disabled by default. In MongoDB 3.6, wire protocol compression becomes enabled by default and the zlib compression algorithm was added as an optional compression with snappy being the default. We recommend snappy, unless a higher level of compression (with added cost) is needed.

It’s important to note that MongoDB tools do not yet support wire protocol compression. This includes mongodump, mongorestore, mongoreplay, etc. As these tools are generally used to move a lot of data, there are significant benefits to be had when using these tools over a non-localhost network.

I created this MongoDB ticket earlier this year to add wire protocol compression support to these tools: https://jira.mongodb.org/browse/TOOLS-1668. Please watch and vote for this improvement if this feature is important to you.

$jsonSchema Schema Validation

A big driver for using MongoDB is its “schema-less”, document-based data model. A drawback to this flexibility is sometimes it can result in incomplete/incorrect data in the database, due to the lack of input checking and sanitization.

The relatively unknown “Schema Validation” was introduced in MongoDB 3.2 to address this risk. This feature allowed a user to define what fields are required and what field-values are acceptable using a simple $or array condition, known as a “validator”.

In MongoDB 3.6, a much-more friendy $jsonSchema format was introduced as a “validator” in Schema Validation. On top of that, the ability to query documents matching a defined $jsonSchema was introduced!

Below is an example of me creating a collection named “test” with the required field “x” that must be the bsonType: “number”:

test1:PRIMARY> db.createCollection("test", {
	validator: {
		$jsonSchema: {
			bsonType: "object",
			required: ["x"],
			properties: {
				x: {
					bsonType: "number",
					description: ”field ‘x’ must be a number"
				}
			}
		}
	}
})
{ "ok" : 1, "operationTime" : Timestamp(1513090298, 1) }

Now when I insert a document that does not contain this criterion (“x” should be a number), I get an error:

test1:PRIMARY> db.test.insert({ x: "abc" })
WriteResult({
	"nInserted" : 0,
	"writeError" : {
		"code" : 121,
		"errmsg" : "Document failed validation"
	}
})

Of course, if my document matches the schema my insert will succeed:

test1:PRIMARY> db.test.insert({ x: 1 })
WriteResult({ "nInserted" : 1 })

To demonstrate $jsonSchema further, let’s perform a .find() query that returns documents matching my defined schema:

test1:PRIMARY> db.test.find({
	$jsonSchema:{
		bsonType: "object",
		required: ["x"],
		properties: {
			x: {
				bsonType: "number",
				description: "must be a number"
			}
		}
	}
})
{ "_id" : ObjectId("5a2fecfd6feb229a6aae374d"), "x" : 1 }

As we can see here, combining the power of the “schema-less” document model of MongoDB with the Schema Validation features is a very powerful combination! Now we can be sure our documents are complete and correct while still offering an extreme amount of developer flexibility.

If data correctness is important to your application, I suggest you implement a Schema Validator at the very start of your application development as implementing validation after data has been inserted is not straightforward.

More on $jsonSchema can be found here: https://docs.mongodb.com/manual/core/schema-validation/#json-schema

DNS SRV Connection

DNS-based Seedlists for connections is a very cool addition to MongoDB 3.6. This allows the server, mongo shell and client drivers (that support the new feature) to use a DNS SRV record to gather a list of MongoDB hosts to connect to. This avoids administrators from having to change seed hosts lists on several servers (usually in an application config) when the host topology changes.

DNS-based seedlists begin with “mongodb+srv://” and have a single DNS SRV record as the hostname.

An example:

mongodb+srv://server.example.com/

Would cause a DNS query to the SRV record ‘_mongodb._tcp.server.example.com’.

On the DNS server, we set the full list of MongoDB hosts that should be returned in this DNS SRV record query. Here is an example DNS response this feature requires:

Record                            TTL   Class    Priority Weight Port  Target
_mongodb._tcp.server.example.com. 86400 IN SRV   0        5      27317 mongodb1.example.com.
_mongodb._tcp.server.example.com. 86400 IN SRV   0        5      27017 mongodb2.example.com.

In this above example the hosts ‘mongodb1’ and ‘mongodb2.example.com’ would be used to connect to the database. If we decided to change the list of hosts, only the DNS SRV record needs to be updated. Neat!

More on this new feature here: https://docs.mongodb.com/manual/reference/connection-string/#connections-dns-seedlist

dropDatabase Wait for Majority

In 3.6 the behavior of ‘dropDatabase’ was changed to wait for a majority of members to drop the database before returning success. This is a great step in the right direction to improve data integrity/correctness.

More on this change here: https://docs.mongodb.com/manual/reference/command/dropDatabase/#behavior

FTDC for mongos

On mongod instances the FTDC (full-time diagnostic capture) feature outputs .bson files to a directory named ‘diagnostics.data’ in the database path (the server dbPath variable). These files are useful for diagnostics, understanding crashes, etc.

On mongos the new FTDC support outputs the .bson files to ‘mongos.diagnostic.data’ beside the mongos log file. You can change the output path for FTDC files with the server parameter diagnosticDataCollectionDirectoryPath.

FTDC output files must be decoded to be read. The GitHub project ‘ftdc-utils’ is a great tool for reading these specially-formatted files, see more about this tool here: https://github.com/10gen/ftdc-utils.

Here is an example of how to decode the FTDC output files. We can follow the same process for mongod as well:

$ cd /path/to/mongos/mongos.diagnostic.data
$ ftdc decode metrics.2017-12-12T14-44-36Z-00000 -o output

Now it decodes the FTDC metrics to the file ‘output’.

listDatabases Filters

Added in MongoDB 3.6, you can now filter the ‘listDatabases‘ server command. Also, a ‘nameOnly’ boolean option was added to only output database names without additional detail.

The filtering of output is controlled by the new ‘listDatabases‘ option ‘filter’. The ‘filter’ variable must be a match-document with any combination of these available fields for filtering:

  1. name
  2. sizeOnDisk
  3. empty
  4. shards

An example filtering by “name” equal to “tim”:

test1:PRIMARY> db.adminCommand({ listDatabases:1, filter: { name: "tim" } })
{
	"databases" : [
		{
			"name" : "tim",
			"sizeOnDisk" : 8192,
			"empty" : false
		}
	],
	"totalSize" : 8192,
	"ok" : 1,
	"operationTime" : Timestamp(1513100396, 1)
}

Here, I am filtering ‘sizeOnDisk’ to find database larger than 30,000 bytes:

test1:PRIMARY> db.adminCommand({ listDatabases:1, filter: { sizeOnDisk: { $gt: 30000 } } })
{
	"databases" : [
		{
			"name" : "admin",
			"sizeOnDisk" : 32768,
			"empty" : false
		},
		{
			"name" : "local",
			"sizeOnDisk" : 233472,
			"empty" : false
		},
		{
			"name" : "test",
			"sizeOnDisk" : 32768,
			"empty" : false
		},
		{
			"name" : "tim",
			"sizeOnDisk" : 32768,
			"empty" : false
		}
	],
	"totalSize" : 331776,
	"ok" : 1,
	"operationTime" : Timestamp(1513100566, 2)
}

This can be really useful to reduce the size of the ‘listDatabases‘ result.

More on this here: https://docs.mongodb.com/manual/reference/command/listDatabases/#dbcmd.listDatabases

Arbiter priority: 0

MongoDB 3.6 changed the arbiter replica set priority to be 0 (zero). As the arbiter’s priority is not considered, this is a more correct value. You’ll notice your replica set configuration is automatically updated when upgrading to MongoDB 3.6.

More on this change here: https://docs.mongodb.com/manual/tutorial/adjust-replica-set-member-priority/#considerations

More on MongoDB 3.6

There are many more changes in this release. It’s important to review these resources below before any upgrade. We always strongly recommend testing functionality in a non-production environment!

Check David Murphy’s blog post on MongoDB 3.6 sessions.

Release Notes: https://docs.mongodb.com/manual/release-notes/3.6/

Compatibility Changes: https://docs.mongodb.com/manual/release-notes/3.6-compatibility/

Conclusion

It is really exciting to see that with each recent major release the MongoDB project is (impressively) tackling both usability/features, while significantly hardening the existing features.

Give your deployment, developers and operations engineers the gift of these new features and optimizations this holiday season/new year! Best wishes in 2018!

by Tim Vaillancourt at December 22, 2017 05:57 PM

This Week in Data with Colin Charles 20: cPanel changes strategy, Percona Live CFP extended

Colin Charles

Colin CharlesJoin Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

I think the biggest news from last week was from cPanel – if you haven’t already read the post, please do – on Being a Good Open Source Community Member: Why we hesitated on MySQL 5.7. cPanel anticipated MariaDB being the eventual replacement for MySQL, based on movements from Red Hat, Wikipedia and Google. The advantage focused on transparency around security disclosure, and the added features/improvements. Today though, “MySQL now consistently matches or outpaces MariaDB when it comes to development and releases, which in turn is increasing the demand on us for providing those upgraded versions of MySQL by our users.” And maybe a little more telling, “when MariaDB 10.2 became stable in May 2017 it included many features found in MySQL 5.7. However, MySQL reached stable nearly 18 months earlier in October 2015.” (emphasis mine).

So cPanel is going forth and supporting MySQL 5.7. They will continue supporting MariaDB Server for the foreseeable future. This really is cPanel ensuring they are responsive to users: “The people using and building database-driven applications are doing so with MySQL in mind, and are hesitant to add support for MariaDB. Responding to our community’s desires is one of the most important things to us, and this is something that we are hearing asked for from our community consistently.”

I, of course, think this is a great move. Users deserve choice. And MySQL has features that are sometimes still not included in MariaDB Server. Have you seen the Complete list of new features in MySQL 5.7? Or my high-level response to a MariaDB Corporation white paper?

I can only hope to see more people think pragmatically like cPanel. Ubuntu as a Linux distribution still does – you get MySQL 5.7 as a default (very unlike the upstream Debian which ships MariaDB Server nowadays). I used to be a proponent of MariaDB Server being everywhere, when it was community-developed, feature-enhanced, and backward-compatible. However, the moment it stopped being a branch and a true fork is the moment where trouble lies for users. I think it was still marginally fine with 10.0, and maybe even 10.1, but the ability to maintain feature parity with enhanced features has long gone. Short of a rebase? But then… what would be different to the already popular branch of MySQL called Percona Server for MySQL?

While there are wins and support from cloud vendors, like Amazon AWS RDS and Microsoft Azure, you’ll notice that they offer both MySQL and MariaDB Server. Google Cloud SQL notably only offers MySQL. IBM may be a sponsor of the MariaDB Foundation, but I don’t see their services like Compose offering anything other than MySQL (with group replication nonetheless!). Platinum member Alibaba Cloud offers MySQL and PostgreSQL. However, Tencent seems to suggest that MariaDB is coming soon? One interesting statistic to watch would be user uptake naturally.

Events

From an events standpoint, the Percona Live 2018 Call for Papers has been extended to January 12, 2018. We expect an early announcement of maybe ten talks in the week of  January 5. Please submit to the CFP. Have you got your tickets yet? Nab them during our Percona Live 2018 super saver registration when they are the best price!

FOSDEM has got Sveta and myself speaking in the MySQL and Friends DevRoom, but we also have good news in the sense that Peter Zaitsev is also going to be at FOSDEM – speaking in the main track. We’ll also have plenty of schwag at the stand.

I think it’s important to take note of the updates to Percona bug tracking: yes, its Jira all the way. Would be good for everyone to start also looking at how the sausage is made.

Dragph, a “distributed fast graph database“, just raised $3m and released 1.0. Have you used it?

On a lighter note, there seems to be a tweet going around by many, so I thought I’d share it here. Merry Christmas and Happy Holidays.

He’s making a database
He’s sorting it twice
SELECT * FROM girls_boys WHERE behaviour = “nice”
SQL Claus is coming to town!

Releases

Link List

Upcoming appearances

  • FOSDEM 2018 – Brussels, Belgium – February 3-4 2018
  • SCALE16x – Pasadena, California, USA – March 8-11 2018

Feedback

I look forward to feedback/tips via e-mail at colin.charles@percona.com or on Twitter @bytebot.

by Colin Charles at December 22, 2017 06:45 AM

December 21, 2017

Peter Zaitsev

Three P’s of a Successful Black Friday: Percona, Pepper Media Holding, and PMM

Successful Black Friday

As we close out the holiday season, let’s look at some data that tells us how to guarantee a successful Black Friday (from a database perspective).

There are certain peak times of the year where companies worldwide hold their breath in the hope that their databases do not become overloaded or unresponsive. A large percentage of yearly profits are achieved in a matter of hours during peak events. It is critical that the database environment remains online and responsive. According to a recent survey, users will not wait more than 2.5 seconds for a site to load before navigating elsewhere. Percona has partnered with many clients over the years to ensure success during these critical events. Our goal is always to provide our clients with the most responsive, stable open-source database environments in order to meet their business needs.

First Stop: Germany

In this blog post, we are going to take a closer look at what happened during Black Friday for a high-demand, high-traffic, business-critical application. Pepper Media Holding runs global deals sites where users post and vote on top deals on products in real-time. To give you a better idea of what the user sees, there is a screenshot below from their Germany mydealz.de branch of Pepper Media Holding.Successful Black Friday

As you can imagine, Black Friday results in a huge spike in traffic and user contribution. In order to ensure success during these crucial times, Pepper Media Holding utilizes Percona’s fully managed service offering. Percona’s Managed Services team has become an extension of Pepper Media Holding’s team by helping plan, prepare, and implement MySQL best-practices across their entire database environment.

Pepper Media Holding and Percona thought it would be interesting to reflect on Black Friday 2017 and how we worked together to flourish under huge spikes in query volume and user connections.

Below is a graph of MySQL query volume for Germany servers supporting the mydealz.de front-end. This graph is taken from Percona’s Managed Service Team’s installation of Percona Monitoring and Management (PMM), which they use to monitor Pepper Media’s environment.

As to be expected, MySQL query volume peaked shortly before and during midnight local time. It also spiked early in the morning as users were waking up. The traffic waned throughout the day. The most interesting data point is the spike from 5 AM to 9 AM which saw an 800% increase from the post-midnight dip. The sustained two-day traffic surge was on average a 200% increase when compared to normal, day-to-day query traffic hitting the database.

For more statistics on how the mydealz.de fared from a front-end and user perspective, visit Pepper Media Holding’s newsroom where Pepper Media has given a breakdown of various statistics related to website traffic during Black Friday.

Next Stop: United Kingdom

Another popular Pepper Media Holding branch is in the United Kingdom – better known as HotUKDeals. HotUKDeals hosts user-aggregated and voted-on deals for UK users. This is the busiest Pepper Media Holding database environment on average. Below is a screenshot of the user interface.

The below graphs are from our Managed Service Team’s Percona Monitoring and Management installation and representative of the UK servers supporting the HotUKDeals website traffic.

The first graph we are taking a look at is MySQL Replication Delay. As you can see, the initial midnight wave of Black Friday deals caused a negligible replica delay. The Percona Monitoring and Management MySQL Replication Delay graph is based on seconds_behind_master which is an integer value only. This means the delay is somewhere between 0 and 1 most of the time. Only once did it go between 1 and 2 over the entire course of Black Friday traffic.

The below graphs highlight the MySQL Traffic seen on the UK servers during the Black Friday traffic spike. One interesting note with this graph is the gradual lead-up to the midnight Black Friday spike. It looks like Black Friday is overstepping its boundaries into Gray Thursday. The traffic spikes here mimic the ones we saw in Germany. There’s an initial spike at midnight on Black Friday and then another spike as shoppers are waking up for their day. The UK servers saw a 361% spike in traffic the morning of Black Friday.

MySQL connections also saw an expected and significant spike during this time. Neglecting to consider max_connections system parameter during an event rush might result in “ERROR 1040 (00000): Too many connections.” However, our CEO, Peter Zaitsev, cautions against absent-mindedly setting this parameter at an unreachable level just to avoid this error. In a blog post, he explained best-practices for this scenario.

The MySQL query graph below shows a 400% spike in MySQL queries during the peak Black Friday morning traffic rush. The average number of queries hitting the database over this two day period is significantly higher than normal – approximately 183%.

Conclusion

Percona reported no emergencies during the Black Friday period for its Managed Service customers – including Pepper Media Holding. We saw similarly high traffic spikes among our customers during this 2017 Black Friday season. I hope that this run-down of a few PMM graphs taken during Pepper Media Holding’s Black Friday traffic period was informative and interesting. Special thanks to Pepper Media Holding for working with us to create this blog post.

Note: Check out our Pepper Media case study on how Percona helps them manage their database environment.

If you would like to further explore the graphs and statistics that Percona Monitoring and Management has to offer, we have a live demo available at https://pmmdemo.percona.com. To discuss how Percona Managed Services can help your database thrive during event-based traffic spikes (and all year round), please call us at +1-888-316-9775 (USA), +44 203 608 6727 (Europe), or have us contact you.

by Barrett Chambers at December 21, 2017 07:54 PM

Jean-Jerome Schmidt

Open Source Databases in 2017 and Trends for 2018

With 2017 quickly coming to a close and 2018 looming on the horizon we wanted to take a minute and reflect on what’s been happening in our industry in the past year and what we are excited about for the future.

Johan Andersson, Severalnines CTO and Co-Founder took a few minutes from working on the newest releases of ClusterControl to talk with us about his thoughts on 2017.

2018 Database Trends and Predictions

As technology moves fast the open source world moves even faster. Here are some predictions from around the web for 2018…

FORBES

  • “In 2017, DevOps suffered from under-budgeting and a perception from management that things that were inexpensive as tools were mostly open source. However, non-standardized adoption and expensive DevOps resources skewed budget. With the realization that open source doesn’t equal free, especially as enterprise-grade support is required, there will be increased awareness of the budget needed for skilled DevOps resources. This barrier should get lowered -- organizations will need a budget for experimentation and failure.”

GITHUB

  • “Data will rule all. Over the last several years, Cloud 1.0 has been about computing in big clouds, while Cloud 2.0 is all about data. This includes data movement and the tools and services that support it, like analytics and machine learning systems. Today all companies are data companies, whether they know it or not. In 2018, so long as teams know how to use it, data will become their greatest asset.”
  • “A decade ago, Linux was a big deal. Now it’s standard. Back in the day, companies like Amazon, Google, and Microsoft were forced to build their own, proprietary tools because no other software existed to meet their needs. Many of these frameworks have since been open sourced—and other open source technologies, like Kubernetes, are becoming integral to developers' workflows. This shift is changing what companies are investing in, making open source software traditional software's biggest competitor.”

OPENSOURCE.COM

  • “Containers gain even more acceptance. Container technology is the approach of packaging pieces of code in a standardized way so they can be "plugged and run" quickly in any environment. Container technology allows enterprises to cut costs and implementation times. While the potential of containers to revolutionize IT infrastructure has been evident for a while, actual container use has remained complex. Container technology is still evolving, and the complexities associated with the technology decrease with every advancement. The latest developments make containers quite intuitive and as easy as using a smartphone, not to mention tuned for today's needs, where speed and agility can make or break a business.”

DBTA.COM

  • “Rapid Kubernetes adoption forms the foundation for multi-cloud deployments:We predict runaway success of Kubernetes, but it is running away with the prize of adoption so fast that this may quickly be more of an observation than a prediction in 2018. So far, however, almost everybody is thinking of Kubernetes as a way of organizing and orchestrating computation in a cloud. Over the next year, we expect Kubernetes to more and more be the way that leading-edge companies organize and orchestrate computation across multiple clouds, both public and private. On premises computation is moving to containers and orchestration style at light speed, but when you can interchangeably schedule work anywhere that it makes sense to do so, you will see the real revolution.”
  • “Fear of cloud lock-in will result in cloud sprawl: As CIOs try to diversify investment in their compute providers, inclusive of their own on-premise capabilities, the diversification will result in data, services and algorithms spreading across multiple clouds. Finding information or code within a single cloud is tough enough. The data silos built from multiple clouds will be deep and far apart, pushing the cost of management onto humans that need to understand the infrastructure.”
ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

Severalnines 2018 Predictions

Several members of our team took a moment to share their thoughts on 2018 and the future of the open source database world…

Vinay Joosery, CEO & Co-Founder

  • Databases in Containers: I think a lot of people have gone from discussing the pros and cons of Docker for databases, and whether it is a good idea at all when it comes to running databases on Docker, to trying it not only in test/dev, but in actual live production!

Alex Yu, VP of Products

  • Cloud and containerized services as the new norm - More companies will start new projects in the cloud with multiple providers and applications will be written with cloud-native architectures in mind.
  • Traditional monolithic applications will continue to give away to more loosely coupled services which are easier to build, deploy, update, scale or move to other cloud and container service providers. Cloud-native services are resilient by nature and will facilitate faster development cycles and feedback loops.
  • Container technologies such as Kubernetes and Docker are already de-facto standard for typical stateless applications/services using "application containers" however databases though are intrinsically stateful and we will hopefully see more use of "system containers" such as LXD coming into the mainstream and gain adoption.
  • This "new world" has an impact on database management and monitoring applications where containerized services come and go frequently. The lifetime of a host is many times longer than a container where the uptime can be measured in only minutes or hours. Host centric management and monitoring will give away to a cloud-native service oriented model where the transient nature is the norm.

Jean-Jérôme Schmidt, VP of Marketing

  • The European Union (EU) is the world’s 2nd largest economy after China and before the US. It’s about to enact a new piece of legislation with far-reaching consequences for anyone doing business involving its residents … and yet it seems to be going almost unnoticed. The European General Data Protection Regulation (GDPR) is due to come into effect in May 2018 and will impact anyone and any business or organisation that deals with and stores EU residents’ personal data in some form or shape. And it will make the organisations that are handling that data responsible for any breaches or misuse. It will therefore be of the highest importance how data is collected, processed and secured. In other words, databases and their infrastructures will be in the spotlight more than ever before and how these database are automated and managed will be crucial for anyone doing business with or within the EU. The GDPR is probably not getting the attention it should because the perception that’s being maintained globally is that the US (and maybe China) are the only large-scale economies worth being concerned with, but the reality is that the EU is the one to be focussed on, particularly next year. So if you’re not sure whether you have your databases and their EU residents data 99.999% under control … contact us ;-)

Ashraf Sharif, Senior Support Engineer

  • We are expecting higher adoption of MySQL 8.0 once it becomes GA. It introduces many notable enhancements like transactional data dictionary, atomic DDL, invisible indexes, common table expression (CTE), windows function and MySQL roles to name some of them. More details at MySQL Documentation page. We are also forecasting a growth in MyRocks storage engine adoption which is already included in Percona Server 5.7 and MariaDB 10.2.
  • MySQL on Docker will be getting much more attention in the coming years, after a number of success stories like Uber and BlablaCar. We've seen many people trying to adapt this technology as a reliable backend data service with the help of in-house automation scripts and Docker orchestration tools. Besides, Docker has announced support for Kubernetes, allowing developers and operators to build apps with Docker and seamlessly test and deploy them using both Docker Swarm and Kubernetes.

Krzysztof Książek, Senior Support Engineer

  • The main new trend that I see today is a move towards column store, analytics databases. MariaDB has it as part of their offering and ClickHouse seems to get traction as a go-to analytics Database engine that works alongside MySQL. ProxySQL's support for ClickHouse also makes it easier for the application to connect to either MySQL or ClickHouse, whatever is needed at that moment. If your dataset is small, you can do analytics in MySQL but there are other tools which do it better - faster and use less disk to store the data.

by fwlymburner at December 21, 2017 10:50 AM

December 20, 2017

Peter Zaitsev

Percona Live 2018 Call for Papers Deadline Extended to January 12, 2018

Percona Live 2018 Call for Papers

Percona Live 2018 Call for PapersPercona is extending the Percona Live 2018 call for papers deadline to January 12, 2018!

Percona’s gift to you this holiday season is the gift of time – submit your speaking topics right up until January 12, 2018!

As the year winds up, we received many requests to extend the Percona Live Open Source Database Conference 2018 call for papers. Since many speakers wanted to submit during the week that they’re planning vacations (from Christmas until New Year’s Day), we realized that December 22 was too soon.

If you haven’t submitted already, please consider doing so. Speaking at Percona Live is a great way to talk about what you’re doing, build up your personal and company brands, and get collaborators to your project. If selected, all speakers receive a full complimentary conference pass.

Percona Live 2018 is the destination to share, learn and explore all pertinent topics related to open source databases. The theme for Percona Live 2018 is “Championing Open Source Databases,” with topics on MySQLMongoDB and other open source databases, including time series databases, PostgreSQL and RocksDB. Session tracks include Developers, Operations, and Business/Case Studies.

Percona Live KeynotesRemember, just like last year, we aren’t looking for just MySQL-ecosystemrelated talks (that includes MariaDB Server and Percona Server for MySQL). We are actively looking for talks around MongoDB, as well as other open source databases (so this is where you can add PostgreSQL, time series databases, graph databases, etc.). That also involves complementary technologies, such as the increasing importance of the cloud and container solutions such as Kubernetes.

Talk about your journey to open source. Describe the technical and business values of moving to or using open source databases. How did you convince your company to make the move? Was there tangible ROI? Share your case studies, best practices and technical knowledge with an engaged audience of open source peers.

We are looking for breakout sessions (25 or 50 minutes long), tutorials (3 hours or 6 hours long), and lightning talks and birds of a feather sessions. Submit as many topics as you think you can deliver well.

The conference itself features one day of tutorials and two days of talks. There will also be exciting keynote talks. Don’t forget that registration is now open, and our Super Saver tickets are the best price you can get (Super Saver tickets are on sale until January 7, 2018).

If your company is interested in sponsoring the conference, please take a look at the sponsorship prospectus.

All in, submit away and remember the Percona Live 2018 call for papers deadline is January 12, 2018. We look forward to seeing you at the conference from April 23-25 2018 in Santa Clara.

by Colin Charles at December 20, 2017 11:46 PM

Updates to Percona Bug Tracking

Percona Bug Tracking

Percona Bug TrackingWe’re completing our move of Percona bug tracking into JIRA, and the drop-dead date is December 28, 2017.

For some time now, Percona has maintained both the legacy Launchpad bug tracking system and a JIRA bug tracking system for some of the newer products. The time has come to consolidate everything into the JIRA bug tracking system.

Assuming everything goes according to schedule, on December 28, 2017, we will copy all bug reports in Launchpad into the appropriate JIRA projects (with the appropriate issue state). The new JIRA issue will link to the original Launchpad issue, and the new JIRA issue link is added to the original Launchpad issue. Once this is done, we will then turn off editing on the Launchpad projects.

Q&A

Which Launchpad projects are affected?
Why are you copying all closed issues from Launchpad?

Copying all Launchpad issues to JIRA enables it to be the one place to search for previously reported issues, instead of having to search for old issues in Launchpad and new issues in JIRA.

What should I do now to prepare?

Go to https://jira.percona.com/ and create an account.

Thanks for reporting bugs, and post any questions in the comments section.

by Peter Schwaller at December 20, 2017 04:28 PM

Jean-Jerome Schmidt

Zero Downtime Network Migration with MySQL Galera Cluster using Relay Node

Galera Cluster’s automatic node provisioning simplifies the complexity of scaling out a database cluster with guaranteed data consistency. SST and IST improve the usability of initial data synchronization without the need to manually backup the database and copy it to the new node. Combine this with Galera's ability to tolerate different network setups (e.g, WAN replication), we can now migrate the database between different isolated networks with zero service disruption.

In this blog post, we are going to look into how to migrate our MySQL Galera Cluster without downtime. We will move the database from Amazon Web Service (AWS) EC2 to Google Cloud Platform (GCP) Compute Engine, with the help of a relay node. Note that we had a similar blog post in the past, but this one uses a different approach.

The following diagram simplifies our migration plan:

Old Site Preparation

Since both sites cannot communicate with each other due to security group or VPC isolation, we need to have a relay node to bridge these two sites together. This node can be located on either site, but must able to connect to one or more nodes on the other side on port 3306 (MySQL), 4444 (SST), 4567 (gcomm) and 4568 (IST). Here is what we already have, and how we will scale the old site:

You can also use an existing Galera node (e.g, the third node) as the relay node, as long as it has connectivity to the other side. The downside is that the cluster capacity will be reduced to two, because one node will be used for SST and relaying the Galera replication stream between sites. Depending on the dataset size and connection between sites, this can introduce database reliability issues on the current cluster.

So, we are going to use a fourth node, to reduce the risk on the current production cluster when syncing to the other side. First, create a new instance in the AWS Dashboard with a public IP address (so it can talk to the outside world) and allow the required Galera communication ports (TCP 3306, 4444, 4567-4568).

Deploy the fourth node (relay node) on the old site. If you are using ClusterControl, you can simply use "Add Node" feature to scale the cluster out (don't forget to setup passwordless SSH from ClusterControl node to this fourth host beforehand):

Ensure the relay node is in sync with the current cluster and is able to communicate to the other side.

From the new site, we are going to connect to the relay node since this is the only node that has connectivity to the outside world.

New Site Deployment

On the new site, we will deploy a similar setup with one ClusterControl node and three-node Galera Cluster. Both sites must use the same MySQL version. Here is our architecture on the new site:

With ClusterControl, the new cluster deployment is just a couple of clicks away and a free feature in the community edition. Go to ClusterControl -> Deploy Database Cluster -> MySQL Galera and follow the deployment wizard:

Click Deploy and monitor the progress under Activity -> Jobs -> Create Cluster. Once done, you should have the following on the dashboard:

At this point, you are having two separate Galera Clusters - 4 nodes at the old site and 3 nodes at the new site.

Connecting Both Sites

On the new site (GCP), pick one node to communicate with the relay node on the old site. We are going to pick galera-gcp1 as the connector to the relay node (galera-aws4). The following diagram illustrates our bridging plan:

The important things to configure are the following parameters:

  • wsrep_sst_donor: The wsrep_node_name of the donor node. On galera-gcp1, set the donor to galera-aws4.
  • wsrep_sst_auth: SST user credentials in username:password format must follow the old site (AWS).
  • wsrep_sst_receive_address: The IP address that will receive SST on the joiner node. On galera-gcp1, set this to the public IP address of this node.
  • wsrep_cluster_address: Galera connection string. On galera-gcp1, add the public IP address of galera-aws4.
  • wsrep_provider_options:
    • gmcast.segment: Default is 0. Set a different integer on all nodes in GCP.
  1. On the relay node (galera-aws4), retrieve the wsrep_node_name:

    $ mysql -uroot -p -e 'SELECT @@wsrep_node_name'
    Enter password:
    +-------------------+
    | @@wsrep_node_name |
    +-------------------+
    | 10.0.0.13         |
    +-------------------+
  2. On galera-gcp1's my.cnf, set wsrep_sst_donor value to the relay node's wsrep_node_name and wsrep_sst_receive_address to the public IP address of galera-gcp1:

    wsrep_sst_donor=10.0.0.13
    wsrep_sst_receive_address=35.197.136.232
  3. On all nodes on GCP, ensure the wsrep_sst_auth value is identical following the old site (AWS) and change the Galera segment to 1 (so Galera knows both sites are in different networks):

    wsrep_sst_auth=backupuser:mysecretP4ssW0rd
    wsrep_provider_options="base_port=4567; gcache.size=512M; gmcast.segment=1"
  4. On galera-gcp1, set the wsrep_cluster_address to include the relay node's public IP address:

    wsrep_cluster_address=gcomm://10.148.0.2,10.148.0.3,10.148.0.4,13.229.247.149

    **Only modify wsrep_cluster_address on galera-gcp1. Don't modify this parameter on galera-gcp2 and galera-gcp3.

  5. Stop all nodes on GCP. If you are using ClusterControl, go to Cluster Actions dropdown -> Stop Cluster. You are also required to turn off automatic recovery at both cluster and node levels, so ClusterControl won't try to recover the failed nodes.

  6. Now the syncing part. Start galera-gcp1. You can see from the MySQL error log on the donor node that SST is initiated between the the relay node (10.0.0.13) using a public address on galera-gcp1 (35.197.136.232):

    2017-12-19T13:58:04.765238Z 0 [Note] WSREP: Initiating SST/IST transfer on DONOR side (wsrep_sst_xtrabackup-v2 --role 'donor' --address '35.197.136.232:4444/xtrabackup_sst//1' --socket '/var/lib/mysql/m
    ysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix ''   '' --gtid 'df23adb8-b567-11e7-8c50-a386c8cc7711:151181')
    2017-12-19T13:58:04.765468Z 5 [Note] WSREP: DONOR thread signaled with 0
            2017-12-19T13:58:15.158757Z WSREP_SST: [INFO] Streaming the backup to joiner at 35.197.136.232 4444
    2017-12-19T13:58:52.512143Z 0 [Note] WSREP: 1.0 (10.0.0.13): State transfer to 0.0 (10.148.0.2) complete.

    Take note that, at this point of time, galera-gcp1 will be flooded with following lines:

    2017-12-19T13:32:47.111002Z 0 [Note] WSREP: (ed66842b, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.0.0.118:4567 timed out, no messages seen in PT3S
    2017-12-19T13:32:48.111123Z 0 [Note] WSREP: (ed66842b, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.0.0.90:4567 timed out, no messages seen in PT3S
    2017-12-19T13:32:50.611462Z 0 [Note] WSREP: (ed66842b, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.0.0.25:4567 timed out, no messages seen in PT3S

    You can safely ignore this warning since galera-gcp1 keeps trying to see the remaining nodes beyond the relay node on AWS.

  7. Once SST on galera-gcp1 completes, ClusterControl on GCE won't be able to connect the database nodes, due to missing GRANTs (existing GRANTs have been overridden after syncing from AWS). So here is what we need to do after SST completes on galera-gcp1:

    mysql> GRANT ALL PRIVILEGES ON *.* TO cmon@'10.148.0.5' IDENTIFIED BY 'cmon' WITH GRANT OPTION;

    Once this is done, ClusterControl will correctly report the state of galera-gcp1 as highlighted below:

  8. The last part is to start the remaining galera-gcp2 and galera-gcp3, one node at a time. Go to ClusterControl -> Nodes -> pick the node -> Start Node. Once all nodes are synced, you should get 7 as the cluster size:

The cluster is now operating on both sites and scaling out is complete.

Decommissioning

Once the migration completes and all nodes are in synced, you can start to switch your application to the new cluster on GCP:

At this point MySQL data is replicated to all nodes until decommissioning. The replication performance will be as good as the farthest node in the cluster permits. The relay node is critical, as it broadcasts writesets to the other side. From the application standpoint, it's recommended to write to only one site at a time, which means you will have to start redirecting reads/writes from AWS and serve them from GCP cluster instead.

To decommission the old database nodes and move to the cluster on GCP, we have to perform a graceful shutdown (one node at a time) on AWS. It is important to shut down the nodes gracefully, since the AWS site holds the majority number of nodes (4/7) for this cluster. Shutting them down all at once will cause the cluster on GCP to go into non-primary state, forcing the cluster to refuse operation. Make sure the last node to shutdown on the AWS side is the relay node.

Don't forget to update the following parameters on galera-gcp1 accordingly:

  • wsrep_cluster_address - Remove the relay node public IP address.
  • wsrep_sst_donor - Comment this line. Let Galera auto pick the donor.
  • wsrep_sst_receive_address - Comment this line. Let Galera auto pick the receiving interface.

Your Galera Cluster is now running on a completely new platform, hosts and network without a second of downtime to your database service during migration. How cool is that?

by ashraf at December 20, 2017 09:28 AM

December 19, 2017

Peter Zaitsev

Webinar Q&A: Percona XtraDB Cluster 101

Percona XtraDB Cluster

Percona XtraDB ClusterIn this blog, we will answer questions from our webinar on Percona XtraDB Cluster 101.

Recently (7 Dec 2017) I presented a webinar about Percona XtraDB Cluster 101. Firstly, thanks to all the attendees: we had a great webinar with quite some interesting questions and feedback.

Through this blog, I’ll answer most of the questions that were raised during the webinar.

Q. How does the need for the acknowledgment from other nodes affect the speed of writes?

A. There are two parts to replication: delivering a transaction (including acknowledgment) and applying the transaction. Generally, the first part is pretty quick and dictated by the network latency. The second part is time-consuming, but happens asynchronously. So acknowledging a transaction from other nodes is not that time-consuming.

Q. How can geo-distributed nodes affect the speed of writes?

A. The longest node dictates cluster performance (in terms of latency). You can’t write faster than the time it takes for a packet to reach the longest node (round-trip-latency). So geo-distribution does affect write performance.

Q. Would you consider Master -> Slave replication in RDS a traditional replication of MySQL? And how easy is it replicating from PXC to RDS if its possible

A. If an application doesn’t need high availability, then a user can explore the MASTER-SLAVE replication. But I would argue that if I am going to spend time booting two servers (MASTER and SLAVE), then why not boot both as MASTER (through Percona XtraDB Cluster). This ensures HA and write-scalability. Percona XtraDB Cluster is flexible for all topologies, and can act as async-master or async-slave too.

Q. Moving forward, is there a plan to deal with version control tools like Flyway that still uses Get locks?

A. Statements like GET_LOCK that establish local locks at the said node are not cluster-safe, so they are blocked with

pxc_strict_mode=ENFORCING
 and not recommended for use. With that said, if the application/user tries to use these statements in a non-conflicting way (with the load directed to single master) it could still work.

Q. With an ASYNC slave, can you use GTID? I know there is a bug(s) that prevent this currently from working 100% in MariaDB (though the bug is close to being fixed – MDEV-10715)?

A. Yes, you can use GTID with async-slave. MariaDB has different implementation of GTID so I am not in a position to comment on the latter part.

Q. Do tables need to have a primary key for the cluster to work?

A. Yes, all tables that you plan to use in a cluster should have a primary key (

pxc_strict_mode=ENFORCING
 enforces this criterion). This is mainly needed for conflict resolution, when the same conflicting workload is executed on multiple-nodes.

Q. What is the best wsrep_sst_method? (for huge database)

A. Percona XtraDB Cluster recommends using XtraBackup. It doesn’t lock the tables for the complete SST life-time, so you can continue to use the node while it is acting as DONOR.

Q. Is the “show processlist” node-specific? Is there an equivalent command to show the whole cluster process list?

A. Yes, show processlist is node specific. There is currently no way to cluster-wide-processlist.

Q. does PXC support partitioned tables?

A. Yes, using InnoDB native partitioning.

Q. These nodes (PXC-nodes) are api nodes or data nodes ?

A. Data nodes.

Q. If cluster went down then everytime it follow SST/IST?

A. It depends. If there is DONOR that has a missing write-set, then the node can rejoin through IST else SST.

Q. Around how much time it will take to join the cluster?

A. The time a node takes to join back depends on the size of the data. Generally, the time for SST is longer than IST. The good part is with 5.7.17+ we have considerably reduced the time for IST, so that a node can join faster than before.

Q. How does IST (incremental state transfer) process affect cluster performance?

A. IST is asynchronous and doesn’t emit FLOW_CONTROL, so cluster can continue to perform as normal. A small slice of DONOR bandwidth is used to send data to the JOINER, but it is not that significant to affect the overall cluster performance.

Q. How do you handle a situation when three simultaneous transactions try to insert auto_increment value?

A. Percona XtraDB Cluster has a concept of wsrep_auto_increment_control that adjust the increment size on each node based on a number of nodes in the cluster. Please check this link for more details.

Q. Imagine that a table A has a trigger on insert that inserts data into another table B. And there are two concurrent transactions: TA inserts into table A (and the trigger makes an insert into B) and TB that inserts the same data directly into B. Will such a conflict – insert from TB and from trigger – be detected?

A. Yes. A transaction can touch multiple data-objects and when the conflict resolution is done, it will check all the objects that transaction is planning to modify before certifying a transaction is safe to apply.

Q. How PXC will make sure of data integrity with parallel processing?

A. Percona XtraDB Cluster has conflict resolution protocol. This protocol is based on FIRST COMMITTER WIN principle that ensures only the first transaction (from a group of a conflicting transaction) commits to cluster.

Q. I’ve created a three node cluster and replication is working. I’d like to copy our production data to the cluster since exporting and importing from MySQL takes a long time. Should I have waited to bootstrap the cluster until the data directory is transferred?

A. If you already have a cluster in place then you are simply adding new tables to the cluster. You can start adding (LOADING) the tables and these tables are immediately replicated to the other nodes of the cluster. An alternative would be to start the first node of the cluster with the pre-loaded data that then becomes cluster state. Other joining nodes copy it over through SST.

Q. Do we have an option to autospinup the compute nodes in a cloud? If PXC will have that option or do we manually need to spinup the Instance and setup the replication?

A. You will have to manually configure it.

Q. Why does XtraBackup not work due bootstrapping but works perfectly after bootstrapping? rsync is working in both cases.

A. Not sure I get the question completely, but XtraBackup works in all scenarios. If you are facing any issue, please log it on launchpad.

Q. As per flow control, one node waits for the other node to be in sync. Won’t there be latency in writing the data?

A. The transaction originated from one node needs to get replicated on other nodes of the cluster. This is what we can call latency and is dictated by network latency. Flow-control is mainly to regulate a scenario wherein one node of the cluster falls way behind other nodes of the cluster.

Q. Can we set up PXC using AWS EC2?

A. Yes.

Once again, thanks for taking time to attend the webinar. If you have more questions, then please post them to the Percona XtraDB Cluster forum here. Also, we have a lot of blogs about Percona XtraDB Cluster. Make sure you check them out here.

by Krunal Bauskar at December 19, 2017 06:40 PM

Percona Monitoring and Management a Trend Setting Product for 2018

DBTA Trend-Setting Products in Data and Information Management for 2018

…but don’t just take our word for it…

Percona Monitoring and Management
DBTA Trend-Setting Products for 2018

The online industry news magazine Database Trends and Applications (DBTA) has included Percona Monitoring and Management in its roundup of Trend-Setting Products in Data and Information Management for 2018.

“…each year, Database Trends and Applications magazine looks for offerings that promise to help organizations derive greater benefit from their data, make better decisions, work more efficiently, achieve greater security, and address emerging challenges.” [December 7, 2018]

As well as including PMM in its selection of products to watch in 2018, DBTA has included a product spotlight penned by Michael Coburn, Product Manager for PMM. This provides a synopsis of the features and benefits of the product for DBAs.

While here at Percona we know that the PMM team are doing excellent work, it’s great they’ve had independent recognition. This news marks a welcome end to a busy and productive year.

The team isn’t resting on its laurels though! We are looking forward to producing further enhancements to PMM throughout 2018.

Watch this space!

About Percona Monitoring and Management

Percona Monitoring and Management (PMM) is a free and open-source platform for managing and monitoring MySQL® and MongoDB® performance. PMM incorporates some best-of-breed open source tools to provide a comprehensive database monitoring and management facility. You can run PMM in your own environment for maximum security and reliability. It provides thorough time-based analysis for MySQL, MariaDB® and MongoDB servers to ensure that your data works as efficiently as possible. Recent developments have included enhanced support for Amazon RDS and Amazon Aurora.

by Lorraine Pocklington at December 19, 2017 03:43 PM

December 18, 2017

Peter Zaitsev

Webinar Wednesday, December 20, 2017: InnoDB Performance Optimization

InnoDB Performance Optimization

InnoDB Performance OptimizationJoin Percona’s, CEO and Co-Founder, Peter Zaitsev as he presents InnoDB Performance Optimization on Wednesday, December 20, 2017, at 11:00 am PST / 2:00 pm EST (UTC-8).

InnoDB is one of the most commonly used storage engines for MySQL and Percona Server for MySQL. It balances high reliability with high performance and is the focus of the majority of storage engine development by the MySQL and Percona Server for MySQL teams.

This webinar looks at InnoDB, including new developments in MySQL 5.7 as well as Percona Server for MySQL. In it, Peter explains how to use it, and many of the configuration options that help you to get the best performance from your application.

Register for the webinar.

Peter ZaitsevPeter Zaitsev, CEO

Peter Zaitsev co-founded Percona and assumed the role of CEO in 2006. As one of the foremost experts on MySQL strategy and optimization, Peter leveraged both his technical vision and entrepreneurial skills to grow Percona from a two-person shop to one of the most respected open source companies in the business. With over 140 professionals in 30 plus countries, Peter’s venture now serves over 3000 customers – including the “who’s who” of internet giants, large enterprises and many exciting startups. Percona was named to the Inc. 5000 in 2013, 2014, 2015 and 2016. Peter was an early employee at MySQL AB, eventually leading the company’s High Performance Group.

A serial entrepreneur, Peter co-founded his first startup while attending Moscow State University where he majored in Computer Science. Peter is a co-author of High Performance MySQL: Optimization, Backups, and Replication, one of the most popular books on MySQL performance. Peter frequently speaks as an expert lecturer at MySQL and related conferences, and regularly posts on the Percona Database Performance Blog. He has also been tapped as a contributor to Fortune and DZone, and his recent ebook Practical MySQL Performance Optimization Volume 1 is one of percona.com’s most popular downloads. 

by Peter Zaitsev at December 18, 2017 11:24 PM

Percona Monitoring and Management 1.5.3 Is Now Available

Percona Monitoring and Management

Percona Monitoring and ManagementPercona announces the release of Percona Monitoring and Management 1.5.3. This release contains fixes for bugs found after the release of Percona Monitoring and Management 1.5.2, as well as some important fixes and improvements not related to the previous release.

Improvements

  • PMM-1874: The read timeout of the proxy server (/prometheus) has been increased from the default of 60 seconds to avoid nxginx gateway timeout error when loading data-rich dashboards.
  • PMM-1863: We improved our handling of temporary Grafana credentials

Bug fixes

  • PMM-1828: On CentOS 6.9, pmm-admin list incorrectly reported that no monitoring services were running.
  • PMM-1842: It was not possible to restart the mysql:queries monitoring service after PMM Client was upgraded from version 1.0.4.
  • PMM-1797: It was not possible to update the CloudWatch data source credentials.
  • PMM-1829: When the user clicked a link in the Query Abstract column, an outdated version of QAN would open.
  • PMM-1836PMM Server installed in a Docker container could not be started if the updating procedure had been temporarily interrupted.
  • PMM-1871: In some cases, RDS instances could not be discovered.
  • PMM-1845: Converted FLUSH SLOW LOGS to FLUSH NO_WRITE_TO_BINLOG SLOW LOGS so that GTID event isn’t created
  • PMM-1816: Fixed a rendering error in Firefox.

by Borys Belinsky at December 18, 2017 10:49 PM

Colin Charles

Percona Live Santa Clara 2018 CFP

Percona Live Santa Clara 2018 call for papers ends fairly soon — December 22 2017. It may be extended, but I suggest getting a topic in ASAP so the conference committee can view everything fairly and quickly. Remember this conference is bigger than just MySQL, so please submit topics on MongoDB, other databases like PostgreSQL, time series, etc., and of course MySQL.

What are you waiting for? Submit TODAY!
(It goes without saying that speakers get a free pass to attend the event.)

by Colin Charles at December 18, 2017 08:29 AM

December 15, 2017

Peter Zaitsev

MongoDB 3.6 Security Improvements

MongoDB 3.6 sorting

MongoDB 3.6 SecurityIn this blog post, we’ll look at MongoDB 3.6 security improvements.

As we’ve already talked about in this series, MongoDB 3.6 has a number of new features in it. But we have talked less about the new security enhancements in this release. The MongoDB 3.6 security features are particularly exciting. Some of these are just moving packaging default into binary default, but others are evidence that MongoDB is growing up and taking in what security professionals have been asking for. We can break things down into two major areas: Network Listening and more restrictive access controls. Hopefully, as a MongoDB user you are excited by this.

Network Listening

This is a purposely vague topic. As you know, MongoDB is just one of the many NoSQL databases that have been hit with hacks, theft and even ransomware events. It is true this was largely due to updated systems not following best practices, but MongoDB’s binaries did nothing to help the situation. Point of fact, both MongoDB Inc and Percona Server for MongoDB ship with configurations that are more restrictive than the binaries. Percona even has a tool to print a warning that it might not be secured if it detects a public IP bound to the machine.

It should be noted for anyone coming from MySQL or the RDBMS world that this differs from having user:password@host ACL restrictions. Even with this setting, any user can connect from any whitelist IP address. So it’s still important to consider a separate password per environment to prevent accidental issues where a developer thinks they are on a test box and instead drop a database in production. However, all is not lost: there are separate improvements on that front also.

You might ask, “How do I configure the bind IP setting?” (especially if you haven’t been). Let’s assume you have some type of NAT in your system, and 10.10.10.10 is some private NAT address that directly maps to a dedicated public one (your DB nodes are in your own data center, and your applications are somewhere else). We will also assume that while you don’t have a VPN, you do have firewall rules such that only your application hosts are allowed into the database. Not perfect, but better than some things we have seen in the news.

To enable listening on that port, you have two methods: command line and configuration file.

Command Line looks like

mongod --bind-ip 10.10.10.10 --fork --logpath /var/log/mongod/mongod.log --port 17001
.

The configuration file, on the other hand, is more of a YAML format:

net:
   port: 17001
   bindIp: 10.10.10.10
   bindIpAll: false

Please note that you should almost never set bindIpAll, as it forces the old behavior of listening to everything. Instead, use a comma-separated list, like “10.10.10.10, 68.82.123.213, 192.168.10.2”.

User Restrictions – CIDR/IP Whitelisting

Just now we talked about how bindIp works. It is for the server as a whole, not per user (something many RDBM systems have long enjoyed). David Murphy discussed this in his MongoDB 3.6 blog, and how MySQL has had it at least since at least 1998. Not only has MongoDB finally added host control to its abilities, but it’s also raised the game using the power of the document model. Typically in the MySQL world, you define a user like:

GRANT ALL PRIVILEGES ON dbTest.* To 'user'@'hostname' IDENTIFIED BY 'password'; 

Not a bad method really, but what if it allowed networks or specific IPs for a single user? This is actually a great first step, but there is more to go. For now, you can only say what sources and destinations a user can map to. You still need to make a user per environment, as you can’t define roles inside of the restriction arrays.

Let me demonstrate with some examples:

rs1:PRIMARY>devUser={
	"user" : "example_devuser",
	"roles" : [
		{
			"role" : "read",
			"db" : "foo"
		}
	],
	"authenticationRestrictions" : [
		{
			"clientSource" : [
				"10.30.0.0/16"
			],
			"serverAddress" : [
				"10.11.0.0/16"
			]
		}
	],
	"pwd" : "changeme"
}
rs1:PRIMARY>prodUser={
	"user" : "example_produser",
	"roles" : [
		{
			"role" : "readWrite",
			"db" : "foo"
		}
	],
	"authenticationRestrictions" : [
		{
			"clientSource" : [
				"10.11.0.0/16"
			],
			"serverAddress" : [
				"10.11.0.0/16"
			]
		}
	],
	"pwd" : "changeme"
}
rs1:PRIMARY> db.createUser(prodUser)
Successfully added user: {
	"user" : "example_produser",
	"roles" : [
		{
			"role" : "readWrite",
			"db" : "foo"
		}
	],
	"authenticationRestrictions" : [
		{
			"clientSource" : [
				"10.11.0.0/16"
			],
			"serverAddress" : [
				"10.11.0.0/16"
			]
		}
	]
}

We strongly suggest you start to use both of these MongoDB 3.6 security features to enable the best possible security by ensuring only the application host can use the application user, and a developer can only use their user. Additionally, look into using Ansible, Chef, or similar tools enable easy of deploying the restrictions.

Hopefully, we can all save a good amount of accidental dropCollections or ensureIndexes being build in production versus development environments.

by Aayushi Mangal at December 15, 2017 04:30 PM

This Week in Data with Colin Charles 19: Percona Live Tickets, Call for Papers and FOSDEM

Colin Charles

Colin CharlesJoin Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

The Percona Live Call For Papers closes on December 22, but why aren’t you submitting already? Don’t wait till the last minute! Look at our broad scope as well. Worth noting that the best prices for tickets are available now until January 7.

FOSDEM is happening in Brussels, Belgium (like it has for the past many years). All I can say is that the schedule is out, and it was very hard to make a choice! See the schedule. In addition, don’t forget to drop by the project table/stand for Percona – yes, we aim to showcase the ecosystem – we’ll be located at Building H.

ACMUG Wrap-up

Over the weekend, I was in Beijing, China for the annual ACMUG gathering, now expanded to two whole days. A special thanks go out to Rikki and Yanwei for helping get me there, and Lixun Peng for being my simultaneous translator. Most of the talks were in Mandarin, but we did have a bit of a foreigner contingent (from Oracle, Manyi Lu, Luis Soares; from Facebook, Yoshinori Matsunobu, and Junyi Luke Lu, and me from Percona), thus there were also talks in English.

It’s clear that there’s a lot of mixed uses these days – not just MySQL but also MongoDB and Hadoop. Another thing that folk are looking into? PingCap’s TiDB. ScyllaDB was also talked about.

One other major takeaway – learning from Chinese DBAs means you learn at scale. For them, a tiny instance may have 3 million users – that’s the whole population of some countries. More has to be done to get exposed to their ideas and how they are solving big issues; expect to have some kind of track at Percona Live!

Releases

Link List

Upcoming Appearances

  • FOSDEM 2018 – Brussels, Belgium – February 3-4 2018
  • SCALE16x – Pasadena, California, USA – March 8-11 2018

Feedback

I look forward to feedback/tips via e-mail at colin.charles@percona.com or on Twitter @bytebot.

by Colin Charles at December 15, 2017 10:26 AM

December 14, 2017

Peter Zaitsev

Percona Server for MySQL 5.7.20-18 Is Now Available

Percona Server for MySQL 5.7.20-18

Percona Server for MySQL 5.7.20-18Percona announces the GA release of Percona Server for MySQL 5.7.20-18 on December 14, 2017. Download the latest version from the Percona web site or the Percona Software Repositories. You can also run Docker containers from the images in the Docker Hub repository.

Based on MySQL 5.7.20, including all the bug fixes in it, Percona Server for MySQL 5.7.20-18 is the current GA release in the Percona Server for MySQL 5.7 series. Percona’s provides completely open-source and free software. Find release details in the 5.7.20-18 milestone at Launchpad.

New Features:
Bugs Fixed:
  • Percona Server 5.7 Docker images did not include TokuDB. Bugs fixed #1682419 and #1699241.
  • If an I/O syscall returned an error during the server shutdown with Thread Pool enabled, a mutex could be left locked. Bug fixed #1702330 (Daniel Black).
  • Dynamic row format feature to support BLOB/VARCHAR in MEMORY tables requires all the key columns to come before any BLOB columns. This requirement, however, was not enforced, allowing creating MEMORY tables in unsupported column configurations, which then crashed or lose data in usage. Bug fixed #1731483.
  • After fixing bug #1668602, bug #1539504, and bug #1313901, CREATE/DROP TEMPORARY TABLE statements were forbidden incorrectly in transactional contexts, including function and trigger calls, even when they required no binary logging at all. Bug fixed #1711781.
  • Running ANALYZE TABLE while a long-running query is accessing the same table in parallel could lead to a situation where new queries on the same table are blocked in a Waiting for table flush state. Fixed by stopping ANALYZE TABLE flushing affected InnoDB and TokuDB tables from the table definition cache. Bug fixed #1704195 (upstream #87065).
  • CREATE TABLE ... LIKE ... did not use the source row_format on target TokuDB table. Bug fixed #76.
  • TokuDB would encode already encoded database name for a directory name. Bug fixed #74.

Other bugs fixed: #1720810, #83, #80, and #75.

MyRocks Changes:
  • RocksDB has implemented a FlushWAL API which improves upon the performance of MySQL 2-phase-commit during binary log group commit flush stage. This feature adds support for using the FlushWAL API in MyRocks and also matches rocksdb_flush_log_at_trx_commit variable with innodb_flush_log_at_trx_commit behavior. To implement this feature rocksdb_manual_wal_flush and rocksdb_concurrent_prepare variables have been implemented.
  • A new rocksdb_force_compute_memtable_stats_cachetime variable has been implemented that cane be used to specify how long the cached value of memtable statistics should be used instead of computing it every time during the query plan analysis.
  • A new rocksdb_large_prefix variable has been implemented which, when enabled, allows index key prefixes longer than 767 bytes (up to 3072 bytes). This option mirrors the innodb_large_prefix. The values for this variable should be the same between master and slave.
  • A new rocksdb_max_background_jobs variable has been implemented to replace rocksdb_base_background_compactions, rocksdb_max_background_compactions, and rocksdb_max_background_flushes variables. This variable specifies the maximum number of background jobs. It automatically decides how many threads to allocate towards flush/compaction. It was implemented to reduce the number of (confusing) options users and can tweak and push the responsibility down to RocksDB level.
  • A new rocksdb_sim_cache_size variable has been implemented to enable the simulated cache. This can be used to figure out the hit/miss rate with a specific cache size without changing the real block cache.
  • Input can be now sorted by the Primary Key during the bulkload by enabling the rocksdb_bulk_load_allow_unsorted variable.
  • A new rocksdb_ignore_unknown_options variable has been implemented, which when enabled (default) allows RocksDB to receive unknown options and not exit.

The release notes for Percona Server for MySQL 5.7.20-18 are available in the online documentation. Please report any bugs on the launchpad bug tracker.

by Hrvoje Matijakovic at December 14, 2017 09:28 PM

MariaDB AB

MariaDB ColumnStore Data Redundancy – A Look Under the Hood

MariaDB ColumnStore Data Redundancy – A Look Under the Hood Ben Thompson Thu, 12/14/2017 - 12:48

In this blog post, we take a close look at  MariaDB ColumnStore data redundancy, a new feature of MariaDB AX. This feature enables you to have highly available storage and automated PM failover when using local disk storage.

MariaDB ColumnStore data redundancy leverages an open source filesystem called GlusterFS, maintained by RedHat. GlusterFS is an open source, distributed file system that provides continued access to data and is capable of scaling very large data. To enable data redundancy you must install and enable GlusterFS prior to running postConfigure. For more information on this topic, refer to Preparing for MariaDB ColumnStore Installation - 1.1.X. Failover is configured automatically by MariaDB ColumnStore, so that if a physical server experiences a service interruption, data is still accessible from another PM node. 

During postConfigure you are prompted to enter number of redundant copies for each dbroot:

Enter Number of Copies [2-N] (2) >

N = Number of PMs. (An actual number is displayed by postConfigure)

On a multi-node install with internal storage the DBRoots are tied directly to a single PM. 

image2.png

 

On a multi-node install with data redundancy, replicated GlusterFS volumes are created for each DBRoot. To users on the outside this appears to be the same as above. Under the hood, a DBRoot is now a gluster volume, where a gluster volume is a collection of gluster bricks that map to directories on the local file system located here:

/usr/local/mariadb/columnstore/gluster/brick(n) (Default). 

This directory will contain the subdirectories brick1 to brick[n], where n = copies configured. Note: Bricks are numbered sequentially on each PM as they are created by MariaDB ColumnStore and are not related to each other or to DBRoot IDs.

Visually:

image5.png
A Three-PM Installation with Data Redundancy Copies = 2

 

In mcsadmin getStorageConfig this is displayed in text form, like this:

Data Redundant Configuration

Copies Per DBroot = 2
DBRoot #1 has copies on PMs = 1 2 
DBRoot #2 has copies on PMs = 2 3 
DBRoot #3 has copies on PMs = 1 3 

The number of copies can be increased as high as the number of PMs. For a three-PM system, that would that would look like this:

image4.png
A Three-PM Installation with Data Redundancy Copies = 3

 

It is important to note that as the number of copies increases, the amount of network resources for distributing redundant data between PMs also increases. Configuration of number of copies should be kept as low as your data-redundancy requirements allow. Alternatively, if hardware configurations allow, a dedicated network can be configured during installation with postConfigure to help offload gluster network data.

MariaDB ColumnStore assigns DBRoots to a PM by using GlusterFS to mount a dbroot to its associated data directory and used as normal.

PM1:

mount -tglusterfs PM1:/dbroot1 /usr/local/mariadb/columnstore/data1

PM2:

mount -tglusterfs PM2:/dbroot2 /usr/local/mariadb/columnstore/data2

PM3:

mount -tglusterfs PM3:/dbroot3 /usr/local/mariadb/columnstore/data3

At this point when a change is made to any files in a data(n) directory, it is copied to the connected brick. Only the assigned bricks are mounted as the logical DBRoots. The unassigned bricks are standby copies waiting for a failover event.

image3.png

Three-PM Data Redundancy Copies = 2

 

A failover occurs when a service interruption is detected from a PM. In a normal local disk installation, data stored on the dbroot for that module would be inaccessible. With data redundancy, a small interruption occurs while the DBRoot is reassigned to the secondary brick. 

In our example system, PM #3 has lost power. PM #1 would be assigned DBRoot3 along with DBRoot1 since it has been maintaining the replica brick for DBroot3. PM #2 will see no change.

image1.png

Three-PM Data Redundancy Copies = 2 & Failure of PM #3

 

When PM #3 returns, data changes for DBRoot3 and DBRoot2 will be synced across bricks for the volumes by GlusterFS. PM #3 returns to operational and DBRoot3 is unmounted from PM #1 and returned to PM #3.

image3.png

Three-PM Data Redundancy Copies = 2 & PM #3 Recovered

 

This is only a simple example meant to illustrate how MariaDB ColumnStore with data redundancy leverages GlusterFS to provide a simple and effective way to keep your data accessible through service interruptions. 

We are excited to offer data redundancy as part of MariaDB ColumnStore 1.1, which is available for download as part of MariaDB AX, an enterprise open source solution for modern data analytics and data warehousing.

Login or Register to post comments

by Ben Thompson at December 14, 2017 05:48 PM

Peter Zaitsev

What Are Your Cloud Data Plans? Help Inform the Community, Take Percona’s Survey

Cloud Data Plans

Cloud Data PlansFill out a quick survey on your cloud data plans.

Today, keeping your enterprise agile and flexible is not just an advantage, it is a requirement. Many of the systems and processes once controlled by businesses onsite are moving offsite to “service” models. This includes Platform as a Service (PaaS), Software as a Service (SaaS), Infrastructure as a Service (IaaS), Database as a Service (DBaaS), etc.

These services are usually referred to as being in the cloud. The enterprise using the service doesn’t maintain or manage the infrastructure of the service in question.

Migrating database workloads to the cloud can be a vital part of improving customer experience, gaining deeper business insights and increasing efficiency. More enterprises are choosing to move data to the cloud in order to make scaling easy, offload resource overhead or control expenses.

Are you looking at moving your database to the cloud in the next year? What’s your reason? Do you know where you want to go? If no, what are your reasons? Concerns?

Percona wants to know about your cloud data plans. Help the community by filling out this quick survey on when, why and how you’d like to migrate data to the cloud.

Add any thoughts or other options in the comments section. We’ll post a follow-up blog with the results!

Fill out the survey now.

by Dave Avery at December 14, 2017 05:27 PM

Demystifying Sequence Numbers (seqno) in Percona XtraDB Cluster

sequence numbers

In this blog post, we’ll look at how sequence numbers work in Percona XtraDB Cluster.

Introduction

Percona XtraDB Cluster uses multiple sequence numbers (seqno), each having a special role to play. Let’s try to understand the significance of each sequence number.

global_seqno

global_seqno

An active Percona XtraDB Cluster cluster has multiple write-sets (transactions) generated from one or more nodes. These transactions then get replicated to other nodes of the cluster. Each write-set/transaction should have a unique identity across the cluster. global_seqno helps serve this identity. Given write-set will have the same global_seqno on all the nodes of the cluster.

We designed the protocol so that each node processes all write-sets, including self-generated write-sets, in same order – thereby skipping the generation of a unique ID through a common entity (also avoiding single point failure). Each node increments its local counter once it receives the write-set from group channel. On restart, this counter is re-initialized based on the state of the cluster as seen by a respective node. In the attached diagram we can see replicated write-sets across the cluster have the same seqno (x, y, z).

local_seqno

local_seqno

Just like global_seqno, there is a local_seqno, but this seqno is local to the said node. It is, of course, incremented to register receipt of write-set. But in addition, it is also incremented for other activities including configuration change, pause activity, etc…

All internal actions like certification are ordered based on local_seqno through LocalMonitor. This is important in that the first transaction to replicate should be first to go through certification. If there is a conflict, then the followup transaction fails and not the first replicated transaction.

local_seqno resets to 0 on restart.

As we can see in the attached example, each write-set has same or different local seqno, as it is really local to that node and there is no cross-node decision based on this seqno.

last_seen_seqno

last_seen_seqno

When a given transaction is about to replicate, last_seen_seqno is set based on last_committed seqno on the said node. In other words, it is like setting a lower watermark for certification ranges. In simpler terms, this transaction has successfully accommodated changes done until this point. If any new conflicting (conflicting based on keys) transactions are added beyond this point, it can result in certification failure.

Here is an example:

  • Say a trx is about to replicate (it doesn’t yet have global_seqno) assigned.
  • trx last_seen_seqno is updated based on the last committed transaction. In this case “a”.
  • trx is successfully replicated and gets a global_seqno (g). In the meantime, trx with global_seqno b,c,d,e,f are also added to the channel and the respective node certifies and adds them to the certification queue (as seen above).
  • The certification algorithm is now trying to certify trx(g). The trx(g) last_seen_seqno is (a), so the range to certify is from b-f. Since b-f write-sets were added after trx(g) was replicated, these write-sets could potentially create certification conflicts with trx(g).
  • trx(g) keys are then compared with the certification queue before it hits a conflicting key at trx(e). Since global_seqno(trx(e)) > last_seen_seqno(trx(g)), it indicates certification conflict. It is also important to note that trx(e) is already accepted, so trx(g) is one that gets rolled back.

So last_seen_seqno helps determine the lower bound of the certification range.

depends_seqno

depends_seqno

Each transaction has a parent transaction based on the data object it modifies. This parent transaction should be applied first. Then the said transaction can be allowed to get processed.

Again, let’s look at an example:

  • “create db” is a base parent trx. (
    depends_seqno=0
    )
  • “create table” is dependent on it. (
    depends_seqno=1
     for this trx with
    global_seqno=2
    )
  • Insert is an independent action and they are not linked to each other. They have a common parent that is trx with
    global_seqno=2
    .
  • The update statement tends to modify a complete table, so closet transaction in the queue acts as the parent (
    depends_seqno=5
    ).

This depends_seqno is set during certification based on the key evaluation, and it dictates the

apply
 order. In turn, it also sets the commit order for the said transaction. Even though apply can run in parallel, it will not allow an update to proceed until insert transaction (with global_seqno=5) is applied. The order will affect the end-result. (The certification queue continues to purge. Say an insert appears after a long interval, and in the meantime, the certification queue has purged.
depends_seqno
 defaults to point to the starting entry of the queue. A purge happens only when the said entry has been committed on all the nodes and it is safe not to link the insert entry with that old historical entry, as the said goal has been achieved.)

Now, what if there is another transaction with

global_seqno=7
 (update mysql.t2). This transaction doesn’t conflict with any of the existing transactions so that it will have
depends_seqno=1
 (first transaction, known transaction in a queue). It can proceed without waiting for (update test.t1) or (insert test.t1) to complete the apply action.

NOTE: I hope this blog helped clarify the basic significance of each seqno. Leave a comment below if you need a more detailed explanation about any of the related aspect.

by Krunal Bauskar at December 14, 2017 03:21 PM

December 13, 2017

Peter Zaitsev

Exciting Pre-Cursors to MongoDB 3.6 Transactions

MongoDB 3.6 Security

MongoDB 3.6 TransactionsIn this blog post, we are going to focus on the pre-cursors to MongoDB 3.6 transactions.

As already discussed in this series, MongoDB 3.6 has a good number of features in it. Many of them center around sessions (which David Murphy already talked about). Some are highly anticipated by customers, whereas others are very case-specific.

We’ll look at the following areas involving MongoDB 3.6 transactions:

  • Oplog Changes
  • Cluster Time
  • Retryable Writes
  • Causal Consistency
  • Two-Phase Drops
  • killSessions

Oplog Changes: Dynamic Oplog

The oplog collection is responsible for holding all the changes received by the primary so that the secondary can replicate them. It is a circular collection that can hold only a certain amount of data. The difference between the first and last command in that collection is called oplog window. The value of this window is the amount of time we can have a secondary offline without performing an initial sync.

This new feature – which is only available when using wiredTiger – doesn’t require any downtime. Now it is possible to resize the oplog online, compared to the rather complicated previous process of taking nodes out of the rotation and manually running commands on them. In MongoDB 3.6, it’s as simple as running the following on a primary: 

db.runCommand({replSetResizeOplog:1, size: 16384})
. Then you’re done! As this is a command, it also replicates to your secondaries automatically in due time, making your operation teams lives much better.

Logical Lamport Clocks, aka “Cluster Time”

The Cluster Time feature solves a known issue that when we have instances’ clocks out of sync. This historically was a big pain, even when people were using NTPd. This new version gossips the cluster time to each other and keeps this internal clock from trusting the machine’s clock. It means it is virtually impossible to go back in time, as all the operations occur sequentially unless you aren’t writing with

w:majority
 (which is now the default).

This begs the question, “Why do we need this now?” The MongoDB community has long desired to have transactions like RDBMs do. This is not to say they are always needed, but the option is nice to have when you do need it. Sessions are the part of this of course, but so is making sure the whole cluster agrees on time so that you can correlate transactions changes across the entire cluster.

We must always remember one of the best points to MongoDB is its ability to scale out via sharding, and all features and expectations should still work in sharding. So Cluster Time helps move users farther in this direction.

Retryable Writes

Retryable Writes make your application more resilient. It helps programmers make sure their writes were committed in the database. Before the advent of these retryable writes, most of the applications use “try and except” to make sure the writes were safe, which doesn’t work for everything! Let’s suppose a server receives a write (update), but the client doesn’t receive the ack that the write was performed (this could be due to a network issue, for example). If using “try and except”, this application re-sends the write. If this update is not idempotent, we can have an unexpected value after this command runs twice. If your scratching your head about how this works, here is a real-world example.

We have a collection of

store.products
, and we want to remove something from the inventory because someone has placed an order. You might want to run something like
db.products.update({_id: ObjectID("507f191e810c19729de860ea"),{qty:{$inc: -1}})
. Now we get to the real issue. The driver sends the request to your replica-set, but oh no! – an election occurred right after we sent the request. How can we tell if the new primary got this change or not? This is what is called a non-deterministic bug: you’re not clear on what your next steps should be. You don’t know what the inventory was so your not sure if one item was removed from it or not.

We used an example here of an election, but it could be a dropped packet, network split, timeout or any number of other issues in theory. This is exactly the issue we had in MongoDB before 3.6, however now this isn’t an issue. As described in sessions, we do know what the sessionID was, so we can simply ask the replica-set again what happened in that ID once we have a primary again. It will let us know if the change was fully applied or rolled back. Now we know exactly what to do giving use a deterministic behavior. Hurray!

Causal Consistency

Causal Consistency takes advantage of “Cluster Time” in order to offer consistent reads from secondaries. Before the MongoDB 3.6, any application that depended on its own writes needed to read from the primary or use the writeConcern to make sure all the secondaries received the write. This feature traded-off speed for consistency, as the subsequent read waits until the secondary is in the same state (virtual clock) as when the write was performed.

Consider this for a second. Causal consistency can be defined as a model that captures the relationships between operations in the system, and guarantees that each process can observe them in common causal order. In other words, all processes in the system agree on the order of the causally related operations. For example, if there is an operation “A” that causes another operation “B”, then causal consistency assures that each of the other processes in the system observes A before observing B. Therefore, causally related operations and concurrent operations are distinguishable in this model.

In more detail, if one write operation influences another write operation, then these two operations are causally related. One relies on the other. Concurrent operations are the ones that are unrelated or causally independent and are not bound by order in the same way. As such, the following condition holds:

  • Write operations that are related are seen by each process of the system in common order
  • Write operations that are NOT related can occur in any order, and can be seen in different orders by the processes in the system

Fundamentally this means causal is always weaker than full sequential consistence. However, it ignores the pain points related to waiting for all operations, and only has to wait for related ones.

Two-Phase Drops

In the past, if you have ever been in a sharded setup you likely ran into a common issue, where you want to drop a sharded collection with some amount of data. You run the command, then try to run sh.shardCollection(…) to recreate to load data into it. Maybe you had a bug where you corrupted the records, and you wanted to fix it and balance all that at the same time. Unfortunately, you get a “collection already sharded” error back, or maybe it already exists in multiple places. How could this be?

In the old system, you would have the drop command run immediately in kind of a UDP fashion or fire and forget. As long as one request returns, it worked. While this is an extreme example, we have seen them before.

Two-phase drops changes how a collection is removed in a replica set/shard, and depends heavily on a new underlying UUID hidden name for each namespace. By dropping with these, it can be more deterministic. Additionally, it means we can move store.products with UUID XXXXXXXXX out of the way and make a new one with UUID YYYYYYYYYY. Then we can lazily cleanup the old location without blocking you loading data, or without locking up your application.

Furthermore, it allows for sharding the new collection while we remove that 1TB from the old collection UUID location. It also can retry deleting with certainty (we are talking about the old version, not the new one). By using the new Cluster Time feature, the collection can be marked as deleted (and be hidden) and will, in fact, be erased only after all the instances acknowledge that command – solving all the issues in one go.

New “killSession” Command

If you had to kill an operation in MongoDB, you know you need to go into the

db.currentOp().inprog
, find it and then run
db.killOp(op_we_found)
. Easy as pie. You might also know that if you use sharding, and we ran a bad query doing a massive table scan, it’s not nearly as easy. We literally have to crawl every single server hunt through all the running processes to find it and kill it, then we have to go to the mongos, and kill it also. This assumes we could even find it. Matching separate threads in replica sets for a scatter-gather is super hard, and not very reliable as a solution.

Now enter the killSession command, which makes it possible to kill a session using the ID given to the clients or found in the mongos layer. It can then automatically trace back to the sessions on each shard, as they all use the same ID for the cluster-wide operations. killSession comes to help administrators to list, manage and kill cluster-wide queries. Not only does this eliminate hours of troubleshooting over the course of a year, but it also keeps your team’s sanity intact.

I hope you find this article on discovering what is new in MongoDB 3.6 transactions interesting. If you have any question or concern, please ping me @AdamoTonete or @percona on twitter.

by Adamo Tonete at December 13, 2017 09:48 PM

MariaDB AB

Atomic Compound Statements

Atomic Compound Statements vaintroub_g Wed, 12/13/2017 - 16:39

Recently, we had a discussion about a hypothetical feature, "START TRANSACTION ON ERROR ROLLBACK", that our users would find very useful. It would allow sending several commands to the server in one batch (a single network packet), and let the server handle the errors. This would combine efficient network use, and atomic execution. It turns out that it is already possible to do this with MariaDB, albeit with a slightly different syntax.


To execute several statements, stmt1;.. stmtN, and rollback on error, we can use MariaDB Server 10.1's compound statement . Put a transaction inside it, combine it with an EXIT HANDLER, and here you are:

BEGIN NOT ATOMIC
  DECLARE EXIT HANDLER FOR SQLEXCEPTION
  BEGIN  
   ROLLBACK;
   RESIGNAL;
  END;
 
  START TRANSACTION;
 
    stmt1;
    ....
    stmtN;

  COMMIT;
END

The statements are executed in a single transaction, which ends with a COMMIT, unless there is an exception, in which case the EXIT HANDLER takes care of ROLLBACK on any error and propagates the original error with RESIGNAL. QED.

The above is quite similar to what the ANSI standard BEGIN ATOMIC would be able to do, if it was implemented. It is not yet, but for the mentioned use case BEGIN NOT ATOMIC could already be helpful.

For illustration, here is a comparison of a conventional "atomic batch" implementation in Java vs using compounds.

Conventional example

Some Java boilerplate, lots of communication between client and server. On the positive side, portable JDBC code.


void atomicBatch(Connection con, Statement stmt, String[] commands) throws SQLException{
    try {
        con.setAutoCommit(false);
       
        for (String command : commands)
            stmt.execute(command);
        con.commit();
    }
    catch(SQLException e) {
        con.rollback();
        throw e;
    }
    finally {
        con.setAutoCommit(true);
    }
}

Compound statement example

Shorter, more efficient network communication. This does not work with MySQL (no compound statement support).


final String ATOMIC_BATCH_PREFIX = "BEGIN NOT ATOMIC DECLARE EXIT HANDLER FOR SQLEXCEPTION  BEGIN  ROLLBACK;   RESIGNAL;  END; START TRANSACTION;";
final String ATOMIC_BATCH_SUFFIX = "; COMMIT; END";

void atomicBatch(Statement stmt, String[] commands) throws SQLException{
   stmt.execute(ATOMIC_BATCH_PREFIX + String.join(";", Arrays.asList(commands)) + ATOMIC_BATCH_SUFFIX);
}

Recently, we had a discussion about a hypothetical feature, "START TRANSACTION ON ERROR ROLLBACK", that our users would find very useful. It would allow sending several commands to the server in one batch (a single network packet), and let the server handle the errors. This would combine efficient network use, and atomic execution. It turns out that it is already possible to do this with MariaDB, albeit with a slightly different syntax.

Login or Register to post comments

by vaintroub_g at December 13, 2017 09:39 PM

5 Simple Steps to Get Started with MariaDB and Tableau

5 Simple Steps to Get Started with MariaDB and Tableau Dipti Joshi Tue, 12/12/2017 - 23:41

For organizations in today’s data-driven economy, easy and fast access to data and insight into data are crucial to stay competitive. In order to leverage the insight from data, companies need easy-to-use and scalable data platforms and visual reporting of the data through front-end business intelligence tools.

MariaDB offers a transactional solution with MariaDB TX and a high-performance analytics solution with MariaDB AX. Both MariaDB TX and MariaDB AX provide an ANSI SQL interface for end users and BI tools. Tableau’s advanced data-visualization tool can interface directly with MariaDB TX and MariaDB AX so BI professionals can confidently use the very same tooling and interface with the same semantics and behavior regardless of whether the data resides in a transactional system or data warehouse. Recently, the Tableau integration with MariaDB was certified.

In this blog post, we show how to connect Tableau to MariaDB TX or MariaDB AX and how to create advanced visualizations in Tableau using data in MariaDB.
 

Step 1: Download the right packages and load your data into MariaDB.

  • Download and install MariaDB TX (or MariaDB AX).

  • Download and install Tableau Desktop.

  • Next populate data in MariaDB. For this exercise's purpose we have populated MariaDB Server with the TPCH dataset.

 

Step 2: Connect from the Tableau desktop to MariaDB.

 

Step 3: Understand the relationship of the database tables through Tableau.

 

Step 4: Create basic visualizations in Tableau against the data in MariaDB.

 

Step 5: Create advanced visualizations in Tableau against the data in MariaDB.


As shown, you can leverage your data assets with the combination of powerful analytic capabilities of MariaDB AX and transactional data in MariaDB TX with the advanced data visualization from Tableau software.

Please leave a comment or contact us for any questions.

 

In this blog post, we show how to connect Tableau to MariaDB TX or MariaDB AX and how to create advanced visualizations in Tableau using data in MariaDB.

Login or Register to post comments

by Dipti Joshi at December 13, 2017 04:41 AM

Billy Mobile Gets Fast Data-Driven Insights With MariaDB ColumnStore and Tableau

Billy Mobile Gets Fast Data-Driven Insights With MariaDB ColumnStore and Tableau guest Tue, 12/12/2017 - 23:30

This is a guest post by Geoff Cleaves, Business Intelligence Manager at Billy Mobile.

At startups, we acutely feel the technology challenges of running a business. The consequences of implementing a technology that doesn’t deliver the anticipated performance, security or ROI can be disastrous. Unlike long-established companies, startups don’t have the luxury of making time-intensive customizations or waiting for vendor-delivered fixes or upgrades.  

About Billy Mobile

Billy Mobile is a fast growing mobile advertising startup. With headquarters in Barcelona and offices in Singapore, Billy Mobile operates exclusively in the mobile arena of the ad tech industry. Billy Mobile is an ad exchange that programmatically connects advertisers with high traffic publishers in a mobile environment.

Our mission is to optimize advertisers’ ROI and to maximize publishers’ income. Billy Mobile’s competitive advantage is located in the optimization and real-time capacity of its technological platform, Active Bx, an automated, in-house developed and exclusively used algorithm capable of creating predictive models to decide when, where and to whom an advertisement will be shown, thereby attaining the highest performance.

The Challenge

Billy Mobile serves approximately 400 million advertisements a day around the world – primarily in Asia and Europe. With this volume, we needed responsive business intelligence (BI) on more than half a billion events a day to glean key insights into the health of our business and to improve customer service.

We needed the right solution that gave us the speed and insight while also allowing for scalability.

We knew we wanted an open source, columnar database. Also, the ability to build out and run queries in parallel for high performance and fast result was extremely important. Unfortunately, many of the open source solutions we researched didn’t give us these features while also giving us the speed we needed.

Latstly, we needed a database technology that could, out of the box, integrate with our BI solution Tableau.

The Solution

Several months ago we deployed MariaDB ColumnStore and have immediately seen results.

With MariaDB ColumnStore, Billy Mobile is able to easily aggregate and continually update approximately 10 million rows of data per hour. MariaDB ColumnStore allows Billy Mobile to accomplish something we never had before - fast, interactive analysis of big data. Using MariaDB ColumnStore with Tableau, we can explore, drill down and filter data, resulting in valuable insights, in less than 10 seconds.

I’m grateful to MariaDB for saving us so much time and giving us a quick ROI. I don’t know of any other open source column-based databases with similar parallelism that you can lay a third party BI software on.

Learn More

Billy Mobile was able to get up and running with MariaDB ColumnStore and Tableau quickly and easily and saw immediate results. Learn more about MariaDB ColumnStore, a core component of MariaDB AX, our enterprise open source solution for modern data analytics and data warehousing. Get started today–download MariaDB AX.

Learn how Billy Mobile deployed MariaDB ColumnStore for fast, interactive analysis of big data that easily integrates with Tableau for data visualization.

Login or Register to post comments

by guest at December 13, 2017 04:30 AM

December 12, 2017

Oli Sennhauser

Galera Cluster and Antivirus Scanner on Linux

Today we had to investigate in a very strange behaviour of IST and SST on a MariaDB Galera Cluster.

The symptom was, that some Galera Cluster nodes took a very long time to start. Up to 7 minutes. So the customer was concluding that the Galera Cluster node does an SST instead of an IST and was asking why the SST happens.

It have to be mentioned here, that the MariaDB error log is very confusing about whether it is an SST or an IST. So the customer was confused and concluded, that MariaDB Galera Cluster was doing an SST instead of IST.

Further confusing was that this behaviour was not consistently on all 3 nodes and not consistently on the 3 stages production, test and integration.

First we had to clear if the Galera node was doing an IST or an SST to exclude problems with Galera Cache or event Bugs in MariaDB Galera Cluster. For this we were running our famous insert_test.sh and did some node restarts with forcing SST and without.

As a Galera Cluster operator you must mandatorily be capable to determine which one of both State Transfers happens from the MariaDB error log:

MariaDB Error Log with IST on Joiner

2017-12-12 22:29:33 140158145914624 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 204013)
2017-12-12 22:29:33 140158426741504 [Note] WSREP: State transfer required: 
        Group state: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:204013
        Local state: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:201439
2017-12-12 22:29:33 140158426741504 [Note] WSREP: New cluster view: global state: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:204013, view# 7: Primary, number of nodes: 3, my index: 2, protocol version 3
2017-12-12 22:29:33 140158426741504 [Warning] WSREP: Gap in state sequence. Need state transfer.
2017-12-12 22:29:33 140158116558592 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '127.0.0.1' --datadir '/home/mysql/database/magal-101-b/data/'  --defaults-file '/home/mysql/database/magal-101-b/etc/my.cnf'  --parent '16426' --binlog '/home/mysql/database/magal-101-b/binlog/laptop4_magal-101-b__binlog' '
2017-12-12 22:29:33 140158426741504 [Note] WSREP: Prepared SST request: rsync|127.0.0.1:4444/rsync_sst
2017-12-12 22:29:33 140158426741504 [Note] WSREP: REPL Protocols: 7 (3, 2)
2017-12-12 22:29:33 140158426741504 [Note] WSREP: Assign initial position for certification: 204013, protocol version: 3
2017-12-12 22:29:33 140158203852544 [Note] WSREP: Service thread queue flushed.
2017-12-12 22:29:33 140158426741504 [Note] WSREP: IST receiver addr using tcp://127.0.0.1:5681
2017-12-12 22:29:33 140158426741504 [Note] WSREP: Prepared IST receiver, listening at: tcp://127.0.0.1:5681
2017-12-12 22:29:33 140158145914624 [Note] WSREP: Member 2.0 (Node B) requested state transfer from 'Node C'. Selected 1.0 (Node C)(SYNCED) as donor.
2017-12-12 22:29:33 140158145914624 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 204050)
2017-12-12 22:29:33 140158426741504 [Note] WSREP: Requesting state transfer: success, donor: 1
2017-12-12 22:29:33 140158426741504 [Note] WSREP: GCache history reset: old(e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:0) -> new(e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:204013)
2017-12-12 22:29:33 140158145914624 [Note] WSREP: 1.0 (Node C): State transfer to 2.0 (Node B) complete.
2017-12-12 22:29:33 140158145914624 [Note] WSREP: Member 1.0 (Node C) synced with group.
WSREP_SST: [INFO] Joiner cleanup. rsync PID: 16663 (20171212 22:29:34.474)
WSREP_SST: [INFO] Joiner cleanup done. (20171212 22:29:34.980)
2017-12-12 22:29:34 140158427056064 [Note] WSREP: SST complete, seqno: 201439
2017-12-12 22:29:35 140158427056064 [Note] WSREP: Signalling provider to continue.
2017-12-12 22:29:35 140158427056064 [Note] WSREP: SST received: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:201439
2017-12-12 22:29:35 140158426741504 [Note] WSREP: Receiving IST: 2574 writesets, seqnos 201439-204013
2017-12-12 22:29:35 140158426741504 [Note] WSREP: IST received: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:204013
2017-12-12 22:29:35 140158145914624 [Note] WSREP: 2.0 (Node B): State transfer from 1.0 (Node C) complete.
2017-12-12 22:29:35 140158145914624 [Note] WSREP: Shifting JOINER -> JOINED (TO: 204534)
2017-12-12 22:29:35 140158145914624 [Note] WSREP: Member 2.0 (Node B) synced with group.
2017-12-12 22:29:35 140158145914624 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 204535)
2017-12-12 22:29:35 140158426741504 [Note] WSREP: Synchronized with group, ready for connections

MariaDB Error Log with SST on Joiner

2017-12-12 22:32:15 139817123833600 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 239097)
2017-12-12 22:32:15 139817401395968 [Note] WSREP: State transfer required: 
        Group state: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:239097
        Local state: 00000000-0000-0000-0000-000000000000:-1
2017-12-12 22:32:15 139817401395968 [Note] WSREP: New cluster view: global state: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:239097, view# 9: Primary, number of nodes: 3, my index: 2, protocol version 3
2017-12-12 22:32:15 139817401395968 [Warning] WSREP: Gap in state sequence. Need state transfer.
2017-12-12 22:32:15 139817094477568 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '127.0.0.1' --datadir '/home/mysql/database/magal-101-b/data/'  --defaults-file '/home/mysql/database/magal-101-b/etc/my.cnf'  --parent '25291' --binlog '/home/mysql/database/magal-101-b/binlog/laptop4_magal-101-b__binlog' '
2017-12-12 22:32:15 139817401395968 [Note] WSREP: Prepared SST request: rsync|127.0.0.1:4444/rsync_sst
2017-12-12 22:32:15 139817401395968 [Note] WSREP: REPL Protocols: 7 (3, 2)
2017-12-12 22:32:15 139817401395968 [Note] WSREP: Assign initial position for certification: 239097, protocol version: 3
2017-12-12 22:32:15 139817178507008 [Note] WSREP: Service thread queue flushed.
2017-12-12 22:32:15 139817401395968 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (e2fbbca5-df26-11e7-8ee2-bb61f8ff3774): 1 (Operation not permitted)
         at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable.
2017-12-12 22:32:15 139817123833600 [Note] WSREP: Member 2.0 (Node B) requested state transfer from 'Node C'. Selected 1.0 (Node C)(SYNCED) as donor.
2017-12-12 22:32:15 139817123833600 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 239136)
2017-12-12 22:32:15 139817401395968 [Note] WSREP: Requesting state transfer: success, donor: 1
2017-12-12 22:32:15 139817401395968 [Note] WSREP: GCache history reset: old(00000000-0000-0000-0000-000000000000:0) -> new(e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:239097)
2017-12-12 22:32:17 139817123833600 [Note] WSREP: 1.0 (Node C): State transfer to 2.0 (Node B) complete.
2017-12-12 22:32:17 139817123833600 [Note] WSREP: Member 1.0 (Node C) synced with group.
WSREP_SST: [INFO] Joiner cleanup. rsync PID: 25520 (20171212 22:32:17.846)
WSREP_SST: [INFO] Joiner cleanup done. (20171212 22:32:18.352)
2017-12-12 22:32:18 139817401710528 [Note] WSREP: SST complete, seqno: 239153
2017-12-12 22:32:18 139817132226304 [Note] WSREP: (ebfd9e9c, 'tcp://127.0.0.1:5680') turning message relay requesting off
2017-12-12 22:32:22 139817401710528 [Note] WSREP: Signalling provider to continue.
2017-12-12 22:32:22 139817401710528 [Note] WSREP: SST received: e2fbbca5-df26-11e7-8ee2-bb61f8ff3774:239153
2017-12-12 22:32:22 139817123833600 [Note] WSREP: 2.0 (Node B): State transfer from 1.0 (Node C) complete.
2017-12-12 22:32:22 139817123833600 [Note] WSREP: Shifting JOINER -> JOINED (TO: 239858)
2017-12-12 22:32:22 139817123833600 [Note] WSREP: Member 2.0 (Node B) synced with group.
2017-12-12 22:32:22 139817123833600 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 239866)
2017-12-12 22:32:22 139817401395968 [Note] WSREP: Synchronized with group, ready for connections

After we cleared that it really was an IST and that it was not a SST because of some other reasons the question rose: Why does an IST of only a few thousand transactions was taking 420 seconds. And this was not always the case...

So we were looking with top at the Donor and the Joiner during IST and we found that on the Donor node the Antivirus software was heavily using CPU (2 x 50%) and otherwise the system was doing nothing for a while and then suddenly started to transfer data over the network (possibly IST?).
Later we found, that the MariaDB datadir (/var/lib/mysql) was not excluded from the Antivirus software. And finally it looks like the Antivirus software was not properly configured by its Master server because the Antivirus software agent was from a cloned VM and not reinitialized. So the Antivirus Master server seems to be confused because there are 2 Antivirus software agents with the same ID.

Another very surprising situation which we did not expect was, that IST was much heavier influenced by the Antivirus software than SST. SST finished in a few seconds while IST took 420 seconds.

Conclusion: Be careful when using Antivirus software in combination with MariaDB Galera Cluster databases and exclude at least all database directories from virus scanning. If you want to be sure to avoid side effects (noisy neighbours) disable the Antivirus software on the database server at all and make sure by other means, that no virus is reaching your precious MariaDB Galera Cluster...

by Shinguz at December 12, 2017 09:51 PM

Jean-Jerome Schmidt

ClusterControl - All the Feature Highlights & Improvements from 2017

With four major releases in 2017 ClusterControl is better than ever at supporting your MySQL, MariaDB, MongoDB & PostgreSQL environments.

When thinking about the features and functions released in 2017 three main themes emerge…

Delivering High Availability

2017 meant the introduction of ProxySQL, a lightweight yet complex protocol-aware proxy that sits between the MySQL clients and server. It also meant improved support for HAProxy and Keepalived and making sure that MySQL and MariaDB can fully utilize them.

Making You More Efficient

From the introduction of the new ClusterControl CLI to dozens of improvements to our UI to the new system to integration with alarms and chatops, ClusterControl now makes it even easier to manage your database environments.

Mixed Environment Support

ClusterControl has always been the system to manage multiple technologies from a single console and have them work together seamlessly. 2017 meant adding support for the latest versions of MariaDB, MongoDB, MySQL, PostgreSQL, Percona Server, and Galera Cluster.

ClusterControl 1.4.0 - January 2017

Announced in January 2017, ClusterControl version 1.4.0 brought several improvements for MySQL Replication and MongoDB. It was also the first version to introduce features for ProxySQL.

With the new version you are now able to deploy a multi-master replication setup in active - standby mode. One master will actively take writes, while the other one is ready to take over writes should the active master fail. From the UI, you can also easily add slaves under each master and reconfigure the topology by promoting new masters and failing over slaves.

Topology reconfigurations and master failovers are not always possible in case of replication problems, for instance errant transactions. In this version ClusterControl checks for issues before any failover or switchover happens. The admin can define whitelists and blacklists of which slaves to promote to master (and vice versa). This makes it easier for admins to customize failover automation in their replication setups.

For MongoDB we extended support, bringing sharded clusters in addition to replica sets. Coupled with this is the ability to retrieve more metrics for monitoring, adding new advisors and providing consistent backups for sharding. With this release, you could now convert a ReplicaSet cluster to a sharded cluster, add or remove shards from a sharded cluster as well as add Mongos/routers to a sharded cluster.

Lastly, we added our initial support for ProxySQL allowing for its deployment onto MySQL Replication setups.

ClusterControl 1.4.1 - April 2017

April was ProxySQL month at Severalnines. ClusterControl 1.4.1 focused almost exclusively on adding additional features and support for this exciting new load balancing technology.

In this version you could now easily configure and manage your ProxySQL deployments with a comprehensive UI. You could create servers, reorientate your setup, create users, set rules, manage query routing, and enable variable configurations. It was now possible to view query analytics for all queries going through the proxy, and e.g. cache any frequent queries in just a click.

ClusterControl 1.4.2 - June 2017

Coined “The DevOps Edition”, version 1.4.2 brought improved support and new features like automated failover for PostgreSQL & MongoDB and included even more features for ProxySQL.

One of the main highlights in this release is the ClusterControl CLI, which allows users who prefer to manage their databases through the command line. All actions, such as deploying a cluster, using the CLI will be visible in the UI and vice versa.

Also included in this release is the new integration system for alarm notification and chatops systems. This new integration with popular incident management and chat services lets you customise the alarms and get alerted in the ops tools you are already using - e.g., Pagerduty, VictorOps, Telegram, Opsgenie and Slack.

ClusterControl 1.5.0 - November 2017

ClusterControl 1.5 provided an array of exciting new backup functionalities to ensure that your data is secure and available whenever disaster strikes. The release also provides expanded PostgreSQL, MariaDB, MySQL NDB Cluster, and ProxySQL support.

This version introduced a new Backup Wizard with new support for AWS & Google Cloud backups, backup verification, Single Database backups and restores, and the ability to create and restore slaves from a backup rather than doing it from the master. Automatic restore testing was an awaited feature, as it is a time consuming task that is often neglected by database administrators.

PostgreSQL got a number of new features in this version including version 10 support, load balancing and virtual IP support with HAProxy and Keepalived, a new backup method, and support for synchronous replication failover.

The version also included support for MariaDB 10.2 and MySQL NDB Cluster 7.5.

ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

If any of these features appeal to you make sure to upgrade or download the latest version of ClusterControl to take advantage of them.

We look forward to providing you even more features to help you deploy, monitor, manage and scale your open source databases further in 2018!

by jj at December 12, 2017 06:19 PM

December 10, 2017

Valeriy Kravchuk

Fun with Bugs #58 - Bug of the Day From @mysqlbugs

In 2013 I had a habit of writing about MySQL bugs on Facebook almost every day. Typical post looked like this one, link to the bug and few words of wondering with a bit of sarcasm.
By the way, check last comments in Bug #68892 mentioned there - the problem of LOST_EVENTS in master's binary log and a way to workaround it still valid as of MySQL 5.7.17.
At that time I often got private messages from colleagues that Facebook is a wrong media for this kind of posts, these posts make MySQL look "buggy" etc, and eventually I was shut up (for a year or so) in a more or less official way by combined efforts of my employer at that time and Oracle MySQL officials.

A year later I started to write about MySQL bugs again on Facebook, but these posts were not regular any more, and maybe became a bit less sarcastic. Years later, I agree that Facebook should better be used for sharing photos of cats, or nice places, or family members, or even for hot political discussions, as these things annoy people much less than MySQL bugs. So, the need for different media for short annoying messages about MySQL bugs is clear to me. I do think the right media is Twitter (it is annoying by itself), and I am present there as @mysqlbugs since December 2012 actually. I am not a big fan of it historically and used to open it mostly to share information about yet another my blog post, but recently (after they allowed to write longer messages there) I decided to pay more attention to it (until its gone entirely and replaced by something else). So, I post a link to one MySQL bug there every day for a week or so, with the #bugoftheday tag. I quickly noticed that the tag was used by others to share photos of nice insects/bugs, but I don't mind for my posts to end up among those, and I am ready to share some of my related photos as well.

To summarize, I am going to write short messages about MySQL bugs regularly on Twitter. I try to write about some bug I recently notices just because it is new, improperly handled or was involved in some customer issue that I worked on. Let me share 5 last bugs mentioned there, so you can decide if it makes sense for you to follow me or #bugoftheday:
  • Bug #88827 - "innodb uses too much space in the PK for concurrent inserts into the same table". Interesting finding by Mark Callaghan during his recent benchmarking efforts. Still "Open".
  • Bug #88788 - "log_bin is not considered correctly an makes binary logging unusable!". This report by Oli Sennhauser (quickly declared a duplicate of my older Bug #75507) got a really nice number, and it's also interesting how efforts to name hosts in a nicely structured manner may play against poor DBA...
  • Bug #88776 - "Issues found by PVS-Studio static analyzer". Related post with the detailed analysis of problems found by the tool was mentioned few days ago by somebody from MariaDB on Slack, so I immediately noted that the bug comes from the same author, Sergey Vasiliev.
  • Bug #88765 - "This bug reporting form has a ridiculously short character limit for the bug syn". The bug report about MySQL bugs database itself. I also hit the limit there way more often than I'd like to. Discussion continues, but I feel that this is not going to be fixed...
  • Bug #87526 - "The output of 'XA recover convert xid' is not useful". One of our customers hit this problem recently. I can only agree with Sveta Smirnova here, "Output of XA RECOVER should show ID which can be used in XA COMMIT statement."
So, if the list above seems useful or interesting, please, follow me on Twitter. Let's continue to have regular fun with MySQL bugs, now using (hopefully) a more appropriate media!

As a side note, if you are interested in older bugs opened this day years ago and still "Verified", please, check this great page by Hartmut! You may find really great reports like Bug #79581- "Error 1064 on selects from Information Schema if routine name has '\0'", by Sveta Smirnova. This bug is 2 years old today...

by Valeriy Kravchuk (noreply@blogger.com) at December 10, 2017 02:59 PM

December 07, 2017

Peter Zaitsev

Hands-On Look at ZFS with MySQL

ZFS with MySQL

ZFS with MySQLThis post is a hands-on look at ZFS with MySQL.

In my previous post, I highlighted the similarities between MySQL and ZFS. Before going any further, I’d like you to be able to play and experiment with ZFS. This post shows you how to configure ZFS with MySQL in a minimalistic way on either Ubuntu 16.04 or Centos 7.

Installation

In order to be able to use ZFS, you need some available storage space. For storage – since the goal here is just to have a hands-on experience – we’ll use a simple file as a storage device. Although simplistic, I have now been using a similar setup on my laptop for nearly three years (just can’t get rid of it, it is too useful). For simplicity, I suggest you use a small Centos7 or Ubuntu 16.04 VM with one core, 8GB of disk and 1GB of RAM.

First, you need to install ZFS as it is not installed by default. On Ubuntu 16.04, you simply need to run:

root@Ubuntu1604:~# apt-get install zfs-dkms zfsutils-linux

On RedHat or Centos 7.4, the procedure is a bit more complex. First, we need to install the EPEL ZFS repository:

[root@Centos7 ~]# yum install http://download.zfsonlinux.org/epel/zfs-release.el7_4.noarch.rpm
[root@Centos7 ~]# gpg --quiet --with-fingerprint /etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux
[root@Centos7 ~]# gpg --quiet --with-fingerprint /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

Apparently, there were issues with ZFS kmod kernel modules on RedHat/Centos. I never had any issues with Ubuntu (and who knows how often the kernel is updated). Anyway, it is recommended that you enable kABI-tracking kmods. Edit the file /etc/yum.repos.d/zfs.repo, disable the ZFS repo and enable the zfs-kmod repo. The beginning of the file should look like:

[zfs]
name=ZFS on Linux for EL7 - dkms
baseurl=http://download.zfsonlinux.org/epel/7.4/$basearch/
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux
[zfs-kmod]
name=ZFS on Linux for EL7 - kmod
baseurl=http://download.zfsonlinux.org/epel/7.4/kmod/$basearch/
enabled=1
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux
...

Now, we can proceed and install ZFS:

[root@Centos7 ~]# yum install zfs

After the installation, I have ZFS version 0.6.5.6 on Ubuntu and version 0.7.3.0 on Centos7. The version difference doesn’t matter for what will follow.

Setup

So, we need a container for the data. You can use any of the following options for storage:

  • A free disk device
  • A free partition
  • An empty LVM logical volume
  • A file

The easiest solution is to use a file, and so that’s what I’ll use here. A file is not the fastest and most efficient storage, but it is fine for our hands-on. In production, please use real devices. A more realistic server configuration will be discussed in a future post. The following steps are identical on Ubuntu and Centos. The first step is to create the storage file. I’ll use a file of 1~GB in /mnt. Adjust the size and path to whatever suits the resources you have:

[root@Centos7 ~]# dd if=/dev/zero of=/mnt/zfs.img bs=1024 count=1048576

The result is a 1GB file in /mnt:

[root@Centos7 ~]# ls -lh /mnt
total 1,0G
-rw-r--r--.  1 root root 1,0G 16 nov 16:50 zfs.img

Now, we will create our ZFS pool, mysqldata, using the file we just created:

[root@Centos7 ~]# modprobe zfs
[root@Centos7 ~]# zpool create mysqldata /mnt/zfs.img
[root@Centos7 ~]# zpool status
  pool: mysqldata
 state: ONLINE
  scan: none requested
config:
        NAME            STATE     READ WRITE CKSUM
        mysqldata       ONLINE       0     0     0
          /mnt/zfs.img  ONLINE       0     0     0
errors: No known data errors
[root@Centos7 ~]# zfs list
NAME        USED  AVAIL  REFER  MOUNTPOINT
mysqldata  79,5K   880M    24K  /mysqldata

If you have a result similar to the above, congratulations, you have a ZFS pool. If you put files in /mysqldata, they are in ZFS.

MySQL installation

Now, let’s install MySQL and play around a bit. We’ll begin by installing the Percona repository:

root@Ubuntu1604:~# cd /tmp
root@Ubuntu1604:/tmp# wget https://repo.percona.com/apt/percona-release_0.1-4.$(lsb_release -sc)_all.deb
root@Ubuntu1604:/tmp# dpkg -i percona-release_*.deb
root@Ubuntu1604:/tmp# apt-get update
[root@Centos7 ~]# yum install http://www.percona.com/downloads/percona-release/redhat/0.1-4/percona-release-0.1-4.noarch.rpm

Next, we install Percona Server for MySQL 5.7:

root@Ubuntu1604:~# apt-get install percona-server-server-5.7
root@Ubuntu1604:~# systemctl start mysql
[root@Centos7 ~]# yum install Percona-Server-server-57
[root@Centos7 ~]# systemctl start mysql

The installation command pulls all the dependencies and sets up the MySQL root password. On Ubuntu, the install script asks for the password, but on Centos7 a random password is set. To retrieve the random password:

[root@Centos7 ~]# grep password /var/log/mysqld.log
2017-11-21T18:37:52.435067Z 1 [Note] A temporary password is generated for root@localhost: XayhVloV+9g+

The following step is to reset the root password:

[root@Centos7 ~]# mysql -p -e "ALTER USER 'root'@'localhost' IDENTIFIED BY 'Mysql57OnZfs_';"
Enter password:

Since 5.7.15, the password validation plugin by defaults requires a length greater than 8, mixed cases, at least one digit and at least one special character. On either Linux distributions, I suggest you set the credentials in the /root/.my.cnf file like this:

[# cat /root/.my.cnf
[client]
user=root
password=Mysql57OnZfs_

MySQL configuration for ZFS

Now that we have both ZFS and MySQL, we need some configuration to make them play together. From here, the steps are the same on Ubuntu and Centos. First, we stop MySQL:

# systemctl stop mysql

Then, we’ll configure ZFS. We will create three ZFS filesystems in our pool:

  • mysql will be the top level filesystem for the MySQL related data. This filesystem will not directly have data in it, but data will be stored in the other filesystems that we create. The utility of the mysql filesystem will become obvious when we talk about snapshots. Something to keep in mind for the next steps, the properties of a filesystem are by default inherited from the upper level.
  • mysql/data will be the actual datadir. The files in the datadir are mostly accessed through random IO operations, so we’ll set the ZFS recordsize to match the InnoDB page size.
  • mysql/log will be where the log files will be stored. By log files, I primarily mean the InnoDB log files. But the binary log file, the slow query log and the error log will all be stored in that directory. The log files are accessed through sequential IO operations. We’ll thus use a bigger ZFS recordsize in order to maximize the compression efficiency.

Let’s begin with the top-level MySQL container. I could have used directly mysqldata, but that would somewhat limit us. The following steps create the filesystem and set some properties:

# zfs create mysqldata/mysql
# zfs set compression=gzip mysqldata/mysql
# zfs set recordsize=128k mysqldata/mysql
# zfs set atime=off mysqldata/mysql

I just set compression to ‘gzip’ (the equivalent of gzip level 6), recordsize to 128KB and atime (the file’s access time) to off. Once we are done with the mysql filesystem, we can proceed with the data and log filesystems:

# zfs create mysqldata/mysql/log
# zfs create mysqldata/mysql/data
# zfs set recordsize=16k mysqldata/mysql/data
# zfs set primarycache=metadata mysqldata/mysql/data
# zfs get compression,recordsize,atime mysqldata/mysql/data
NAME                  PROPERTY     VALUE     SOURCE
mysqldata/mysql/data  compression  gzip      inherited from mysqldata/mysql
mysqldata/mysql/data  recordsize   16K       local
mysqldata/mysql/data  atime        off       inherited from mysqldata/mysql

Of course, there are other properties that could be set, but let’s keep things simple. Now that the filesystems are ready, let’s move the files to ZFS (make sure you stopped MySQL):

# mv /var/lib/mysql/ib_logfile* /mysqldata/mysql/log/
# mv /var/lib/mysql/* /mysqldata/mysql/data/

and then set the real mount points:

# zfs set mountpoint=/var/lib/mysql mysqldata/mysql/data
# zfs set mountpoint=/var/lib/mysql-log mysqldata/mysql/log
# chown mysql.mysql /var/lib/mysql /var/lib/mysql-log

Now we have:

# zfs list
NAME                   USED  AVAIL  REFER  MOUNTPOINT
mysqldata             1,66M   878M  25,5K  /mysqldata
mysqldata/mysql       1,54M   878M    25K  /mysqldata/mysql
mysqldata/mysql/data   890K   878M   890K  /var/lib/mysql
mysqldata/mysql/log    662K   878M   662K  /var/lib/mysql-log

We must adjust the MySQL configuration accordingly. Here’s what I put in my /etc/my.cnf file (/etc/mysql/my.cnf on Ubuntu):

[mysqld]
datadir=/var/lib/mysql
innodb_log_group_home_dir = /var/lib/mysql-log
innodb_doublewrite = 0
innodb_checksum_algorithm = none
slow_query_log = /var/lib/mysql-log/slow.log
log-error = /var/lib/mysql-log/error.log
server_id = 12345
log_bin = /var/lib/mysql-log/binlog
relay_log=/var/lib/mysql-log/relay-bin
expire_logs_days=7
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
pid-file=/var/run/mysqld/mysqld.pid

On Centos 7, selinux prevented MySQL from accessing files in /var/lib/mysql-log. I had to perform the following steps:

[root@Centos7 ~]# yum install policycoreutils-python
[root@Centos7 ~]# semanage fcontext -a -t mysqld_db_t "/var/lib/mysql-log(/.*)?"
[root@Centos7 ~]# chcon -Rv --type=mysqld_db_t /var/lib/mysql-log/

I could have just disabled selinux since it is a test server, but if I don’t get my hands dirty on selinux once in a while with semanage and chcon I will not remember how to do it. Selinux is an important security tool on Linux (but that’s another story).

At this point, feel free to start using your test MySQL database on ZFS.

Monitoring ZFS

To monitor ZFS, you can use the zpool command like this:

[root@Centos7 ~]# zpool iostat 3
              capacity     operations     bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
mysqldata   19,6M   988M      0      0      0    290
mysqldata   19,3M   989M      0     44      0  1,66M
mysqldata   23,4M   985M      0     49      0  1,33M
mysqldata   23,4M   985M      0     40      0   694K
mysqldata   26,7M   981M      0     39      0   561K
mysqldata   26,7M   981M      0     37      0   776K
mysqldata   23,8M   984M      0     43      0   634K

This shows the ZFS activity while I was loading some data. Also, the following command gives you an estimate of the compression ratio:

[root@Centos7 ~]# zfs get compressratio,used,logicalused mysqldata/mysql
NAME             PROPERTY       VALUE  SOURCE
mysqldata/mysql  compressratio  4.10x  -
mysqldata/mysql  used           116M   -
mysqldata/mysql  logicalused    469M   -
[root@Centos7 ~]# zfs get compressratio,used,logicalused mysqldata/mysql/data
NAME                  PROPERTY       VALUE  SOURCE
mysqldata/mysql/data  compressratio  4.03x  -
mysqldata/mysql/data  used           67,9M  -
mysqldata/mysql/data  logicalused    268M   -
[root@Centos7 ~]# zfs get compressratio,used,logicalused mysqldata/mysql/log
NAME                 PROPERTY       VALUE  SOURCE
mysqldata/mysql/log  compressratio  4.21x  -
mysqldata/mysql/log  used           47,8M  -
mysqldata/mysql/log  logicalused    201M   -

In my case, the dataset compresses very well (4x). Another way to see how files are compressed is to use ls and du. ls returns the actual uncompressed size of the file, while du returns the compressed size. Here’s an example:

[root@Centos7 mysql]# -lah ibdata1
-rw-rw---- 1 mysql mysql 90M nov 24 16:09 ibdata1
[root@Centos7 mysql]# du -hs ibdata1
14M     ibdata1

I really invite you to further experiment and get a feeling of how ZFS and MySQL behave together.

Snapshots and backups

A great feature of ZFS that work really well with MySQL are snapshots. A snapshot is a consistent view of the filesystem at a given point in time. Normally, it is best to perform a snapshot while a flush tables with read lock is held. That allows you to record the master position, and also to flush MyISAM tables. It is quite easy to do that. Here’s how I create a snapshot with MySQL:

[root@Centos7 ~]# mysql -e 'flush tables with read lock;show master status;! zfs snapshot -r mysqldata/mysql@my_first_snapshot'
+---------------+-----------+--------------+------------------+-------------------+
| File          | Position  | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+---------------+-----------+--------------+------------------+-------------------+
| binlog.000002 | 110295083 |              |                  |                   |
+---------------+-----------+--------------+------------------+-------------------+
[root@Centos7 ~]# zfs list -t snapshot
NAME                                     USED  AVAIL  REFER  MOUNTPOINT
mysqldata/mysql@my_first_snapshot          0B      -    24K  -
mysqldata/mysql/data@my_first_snapshot     0B      -  67,9M  -
mysqldata/mysql/log@my_first_snapshot      0B      -  47,8M  -

The command took about 1s. The only time where such commands would take more time is when there are MyISAM tables with a lot of pending updates to the indices, or when there are long running transactions. You probably wonder why the “USED” column reports 0B. That’s simply because there were no changes to the filesystem since the snapshot was created. It is a measure of the amount of data that hasn’t been free because the snapshot requires the data. Said otherwise, it how far the snapshot has diverged from its parent. You can access the snapshot through a clone or through ZFS as a file system. To access the snapshot through ZFS, you have to set the snapdir parameter to “visible, ” and then you can see the files. Here’s how:

[root@Centos7 ~]# zfs set snapdir=visible mysqldata/mysql/data
[root@Centos7 ~]# zfs set snapdir=visible mysqldata/mysql/log
[root@Centos7 ~]# ls /var/lib/mysql-log/.zfs/snapshot/my_first_snapshot/
binlog.000001  binlog.000002  binlog.index  error.log  ib_logfile0  ib_logfile1

The files in the snapshot directory are read-only. If you want to be able to write to the files, you first need to clone the snapshots:

[root@Centos7 ~]# zfs create mysqldata/mysqlslave
[root@Centos7 ~]# zfs clone mysqldata/mysql/data@my_first_snapshot mysqldata/mysqlslave/data
[root@Centos7 ~]# zfs clone mysqldata/mysql/log@my_first_snapshot mysqldata/mysqlslave/log
[root@Centos7 ~]# zfs list
NAME                        USED  AVAIL  REFER  MOUNTPOINT
mysqldata                   116M   764M    26K  /mysqldata
mysqldata/mysql             116M   764M    24K  /mysqldata/mysql
mysqldata/mysql/data       67,9M   764M  67,9M  /var/lib/mysql
mysqldata/mysql/log        47,8M   764M  47,8M  /var/lib/mysql-log
mysqldata/mysqlslave         28K   764M    26K  /mysqldata/mysqlslave
mysqldata/mysqlslave/data     1K   764M  67,9M  /mysqldata/mysqlslave/data
mysqldata/mysqlslave/log      1K   764M  47,8M  /mysqldata/mysqlslave/log

At this point, it is up to you to use the clones to spin up a local slave. Like for the snapshots, the clone only grows in size when actual data is written to it. ZFS records that haven’t changed since the snapshot was taken are shared. That’s a huge space savings. For a customer, I once wrote a script to automatically create five MySQL slaves for their developers. The developers would do tests, and often replication broke. Rerunning the script would recreate fresh slaves in a matter of a few minutes. My ZFS snapshot script and the script I wrote to create the clone based slaves are available here: https://github.com/y-trudeau/Yves-zfs-tools

Optional features

In the previous post, I talked about a SLOG device for the ZIL and the L2ARC, a disk extension of the ARC cache. If you promise to never use the following trick in production, here’s how to speed MySQL on ZFS drastically:

[root@Centos7 ~]# dd if=/dev/zero of=/dev/shm/zil_slog.img bs=1024 count=131072
131072+0 enregistrements lus
131072+0 enregistrements écrits
134217728 octets (134 MB) copiés, 0,373809 s, 359 MB/s
[root@Centos7 ~]# zpool add mysqldata log /dev/shm/zil_slog.img
[root@Centos7 ~]# zpool status
  pool: mysqldata
 state: ONLINE
  scan: none requested
config:
        NAME                     STATE     READ WRITE CKSUM
        mysqldata                ONLINE       0     0     0
          /mnt/zfs.img           ONLINE       0     0     0
        logs
          /dev/shm/zil_slog.img  ONLINE       0     0     0
errors: No known data errors

The data in the SLOG is critical for ZFS recovery. I performed some tests with virtual machines, and if you crash the server and lose the SLOG you may lose all the data stored in the ZFS pool. Normally, the SLOG is on a mirror in order to lower the risk of losing it. The SLOG can be added and removed online.

I know I asked you to promise to never use an shm file as SLOG in production. Actually, there are exceptions. I would not hesitate to temporarily use such a trick to speed up a lagging slave. Another situation where such a trick could be used is with Percona XtraDB Cluster. With a cluster, there are multiple copies of the dataset. Even if one node crashed and lost its ZFS filesystems, it could easily be reconfigured and reprovisioned from the surviving nodes.

The other optional feature I want to cover is a cache device. The cache device is what is used for the L2ARC. The content of the L2ARC is compressed as the original data is compressed. To add a cache device (again an shm file), do:

[root@Centos7 ~]# dd if=/dev/zero of=/dev/shm/l2arc.img bs=1024 count=131072
131072+0 enregistrements lus
131072+0 enregistrements écrits
134217728 octets (134 MB) copiés, 0,272323 s, 493 MB/s
[root@Centos7 ~]# zpool add mysqldata cache /dev/shm/l2arc.img
[root@Centos7 ~]# zpool status
  pool: mysqldata
 state: ONLINE
  scan: none requested
config:
    NAME                     STATE     READ WRITE CKSUM
    mysqldata                ONLINE       0     0     0
      /mnt/zfs.img           ONLINE       0     0     0
    logs
      /dev/shm/zil_slog.img  ONLINE       0     0     0
    cache
      /dev/shm/l2arc.img     ONLINE       0     0     0
errors: No known data errors

To monitor the L2ARC (and also the ARC), look at the file: /proc/spl/kstat/zfs/arcstats. As the ZFS filesystems are configured right now, very little will go to the L2ARC. This can be frustrating. The reason is that the L2ARC is filled by the elements evicted from the ARC. If you recall, we have set primarycache=metatdata for the filesystem containing the actual data. Hence, in order to get some data to our L2ARC, I suggest the following steps:

[root@Centos7 ~]# zfs set primarycache=all mysqldata/mysql/data
[root@Centos7 ~]# echo 67108864 > /sys/module/zfs/parameters/zfs_arc_max
[root@Centos7 ~]# echo 3 > /proc/sys/vm/drop_caches
[root@Centos7 ~]# grep '^size' /proc/spl/kstat/zfs/arcstats
size                            4    65097584

It takes the echo command to drop_caches to force a re-initialization of the ARC. Now, InnoDB data starts to be cached in the L2ARC. The way data is sent to the L2ARC has many tunables, which I won’t discuss here. I chose 64MB for the ARC size mainly because I am using a low memory VM. A size of 64MB is aggressively small and will slow down ZFS if the metadata doesn’t fit in the ARC. Normally you should use a larger value. The actual good size depends on many parameters like the filesystem system size, the number of files and the presence of a L2ARC. You can monitor the ARC and L2ARC using the arcstat tool that comes with ZFS on Linux (when you use Centos 7). With Ubuntu, download the tool from here.

Removal

So the ZFS party is over? We need to clean up the mess! Let’s begin:

[root@Centos7 ~]# systemctl stop mysql
[root@Centos7 ~]# zpool remove /dev/shm/l2arc.img
[root@Centos7 ~]# zpool remove mysqldata /dev/shm/zil_slog.img
[root@Centos7 ~]# rm -f /dev/shm/*.img
[root@Centos7 ~]# zpool destroy mysqldata
[root@Centos7 ~]# rm -f /mnt/zfs.img
[root@Centos7 ~]# yum erase spl kmod-spl libzpool2 libzfs2 kmod-zfs zfs

The last step is different on Ubuntu:

root@Ubuntu1604:~# apt-get remove spl-dkms zfs-dkms libzpool2linux libzfs2linux spl zfsutils-linux zfs-zed

Conclusion

With this guide, I hope I provided a positive first experience in using ZFS with MySQL. The configuration is simple, and not optimized for performance. However, we’ll look at more realistic configurations in future posts.

by Yves Trudeau at December 07, 2017 10:37 PM

Percona Server for MySQL 5.5.58-38.10 is Now Available

percona server 5.5.58-38.10

Percona Server for MySQL 5.5.58-38.10Percona announces the release of Percona Server for MySQL 5.5.58-38.10 on December 7, 2017. Based on MySQL 5.5.58, including all the bug fixes in it, Percona Server for MySQL 5.5.58-38.10 is now the current stable release in the 5.5 series.

Percona Server for MySQL is open-source and free. You can find release details in the 5.5.58-38.10 milestone on Launchpad. Downloads are available here and from the Percona Software Repositories.

New Features:
  • Percona Server packages are now available for Ubuntu 17.10 (Artful).
Bugs Fixed:
  • If an I/O syscall returned an error during the server shutdown with Thread Pool enabled, a mutex could be left locked. Bug fixed #1702330 (Daniel Black).
  • Dynamic row format feature to support BLOB/VARCHAR in MEMORY tables requires all the key columns to come before any BLOB columns. This requirement however was not enforced, allowing creating MEMORY tables in unsupported column configurations, which then crashed or lose data in usage. Bug fixed #1731483.

Other bugs fixed: #1729241.

Find the release notes for Percona Server for MySQL 5.5.58-38.10 in our online documentation. Report bugs on the launchpad bug tracker.

by Hrvoje Matijakovic at December 07, 2017 07:22 PM

MariaDB AB

MariaDB ColumnStore Write API

MariaDB ColumnStore Write API linuxjedi Thu, 12/07/2017 - 02:19

There are many great unique features of MariaDB ColumnStore, one of which is the speed in which you can get data from CSV files into MariaDB ColumnStore using the tool ‘cpimport’. But what if your data is in another format? Or you wish to stream data into MariaDB ColumnStore from another application? This is where MariaDB ColumnStore’s new write API comes into play.

The MariaDB ColumnStore write API is a new C++ API which lets you inject data directly into MariaDB ColumnStore’s WriteEngine using a series of simple calls. This allows you to easily write custom data injection tools that are much faster than using the SQL interface. If you are a Python or Java developer, then we also bundle in wrappers for those languages.

We designed the API to be familiar to users of ORM ways of accessing the database, we have the function setColumn() which is used to set a column in a row you are going to write, writeRow() to store the row as well as commit() and rollback() functions.

This is an example of a simple application that will write to a MariaDB ColumnStore that has just two integer columns. The full source code for this can be found in the example/basic_bulk_insert.cpp file in the API’s source code:

int main(void)
{
    mcsapi::ColumnStoreDriver* driver = nullptr;
    mcsapi::ColumnStoreBulkInsert* bulk = nullptr;
    try {
        driver = new mcsapi::ColumnStoreDriver();

The ColumnStoreDriver class will automatically discover the MariaDB ColumnStore cluster by trying to find the Columnstore.xml which is in every module, there is also an optional parameter to specify a location for this xml file. It will throw an error if this cannot be found. You can copy the xml file to another server and use it there as long as that server can talk to all your PM nodes.

        bulk = driver->createBulkInsert("test", "t1", 0, 0);

This creates an instance of a bulk insert class from the driver class and sets everything up as required. You can see that we are writing to the table “test.t1”. The API can create many bulk insert objects from a single driver object. Each bulk insert object should be considered a database transaction.

        for (int i = 0; i < 1000; i++)
        {
            bulk->setColumn(0, (uint32_t)i);
            bulk->setColumn(1, (uint32_t)1000 - i);
            bulk->writeRow();
        }

This is a very simple ‘for’ loop which sets an integer for the first and second column of the table and then stores that row. Note that writeRow() is designed to not immediately send data to ColumnStore for performance. It will instead buffer 100,000 rows or wait for a commit() before it actually sends the data.

            bulk->commit();

After we have set these 1000 rows we ask the MariaDB ColumnStore API to commit the data. In this example the data would be sent to the MariaDB ColumnStore installation along with the commit() since we only have 1000 rows. At this point the ‘bulk’ object cannot be used to send any more data. It can only be used for retrieving summary information.

    } catch (mcsapi::ColumnStoreError &e) {
            std::cout << "Error caught: " << e.what() << std::endl;
    }
    delete bulk;
    delete driver;
}

Finally we have a little bit of error handling and cleanup.

The data is written using the same atomic methods as ‘cpimport’ by appending new blocks of data to the column files and moving an atomic “High Water Mark” block pointer upon commit. This means that select queries are not blocked during the insert process and do not see the data until after it has been committed.

There are several more advanced features of the API which you can find in the documentation. The API is an Open Source project and the source code can be easily obtained via GitHub. The API has been released alongside MariaDB ColumnStore 1.1 and you can get it from our download page along with other components that make up MariaDB AX, our modern data warehousing solution.

There are many great unique features of MariaDB ColumnStore, one of which is the speed in which you can get data from CSV files into MariaDB ColumnStore using the tool ‘cpimport’. But what if your data is in another format? Or you wish to stream data into MariaDB ColumnStore from another application? This is where MariaDB ColumnStore’s new write API comes into play.

Login or Register to post comments

by linuxjedi at December 07, 2017 07:19 AM

December 06, 2017

Peter Zaitsev

MongoDB 3.6 Community Is Here!

MongoDB 3.6 Community

MongoDB 3.6 CommunityBy now you surely know MongoDB 3.6 Community became generally available on Dec 5, 2017. Of course, this is great news: it has some big ticket items that we are all excited about! But I want to also talk about my general thoughts on this release.

It is always a good idea for your internal teams to study and consider new versions. This is crucial for understanding if and when you should start using it. After deciding to use it, there is the question of if you want your developers using the new features (or are they not suitably implemented yet to be used)?

So what is in MongoDB 3.6 Community? Check it out:

  • Sessions
  • Change Streams
  • Retryable Writes
  • Security Improvement
  • Major love for Arrays in Aggregation
  • A better balancer
  • JSON Schema Validation
  • Better Time management
  • Compass is Community
  • Major WiredTiger internal overhaul

As you can see, this is an extensive list. But there are 1400+ implemented Jira tickets just on the server itself (not even in the WiredTigers project).

To that end, I thought we should break my review into a few areas. We will have blog posts out soon covering these areas in more depth. This blog is more about my thoughts on the topics above.

Expected blogs (we will link to them as they become available):

  • Change Streams –  Nov 11 2017
  • Sessions
  • Transactions and new functions
  • Aggregation improvements
  • Security Controls to use ASAP
  • Other changes from diagnostics to Validation

Today let’s quickly recap the above areas.

Sessions

We will have a blog on this (it has some history). This move has been long-awaited by anyone using MongoDB before 2.4. There were connection changes in that release that made it complicated for load balancers due to the inability to “re-attach” to the same session.  If you were not careful in 2.4+, you could easily use a load-balancer and have very odd behavior: from broken to invisibly incomplete getMores (big queries).

Sessions aim is to change this. Now, the client drivers know about the internal session to the database used for reads and writes. Better yet, MongoDB tracks these sessions so even if an election occurs, when your drive fails over so will the session. For anyone who’s applications handled fail-overs badly, this is an amazing improvement. Some of the other new features that make 3.6 a solid release require this new behavior.

Does this mean this solution is perfect and works everywhere? No! It, like newer features we have seen, leave MMAPv1 out in the cold due to its inability without major re-work to support logic that is so native to Wired Tiger. Talking with engine engineers, it’s clear that some of the logic behind the underlying database snapshots and rollbacks added here can cause issues (which we will talk about more in the transactions blog).

Change streams

As one of the most talked about (but most specialized) features, I can see its appeal in a very specific use case (but it is rather limited). However, I am getting ahead of myself! Let’s talk about what it is first and where it came from.

Before this feature, people streamed data out of MySQL and MongoDB into Elastic and Hadoop stacks. I mention MySQL, as this was the primer for the initial method MongoDB used. The tools would read the MySQL binlogs – typically saved off somewhere – and then apply those operations to some other technology. When they went to do the same thing in MongoDB, there was a big issue: if writes are not sent to the majority of the nodes, it can cause a rollback. In fact, such rollbacks were not uncommon. The default was w:1 (meaning the primary only needed to have the write), which resulted in data existing in Elastic that had been removed from MongoDB. Hopefully, everyone can see the issue here, and why a better solution was needed than just reading the oplog.

Enter

$changeStream
, which in the shell has a helper called
.watch()
 . This is a method that uses a multi-node consistent read to ensure the data is on the majority of nodes before the command returns the data in a tailable cursor. For this use case, this is amazing as it allows the data replicating tool much more assurance that data is not going to vanish. 
$changeStream
 is not without limits: if we have 10k collections and we want to watch them all, this is 10k separate operations and cursors. This puts a good deal of strain on the systems, so MongoDB Inc. suggests you do this on no more than 1000 namespaces at a time to prevent overhead issues.

Sadly it is not currently possible to take a mongod-wide snapshot to support this under the hood, as this is done on each namespace to implement the snapshot directly inside WiredTiger’s engine. So for anyone with a very large collection count, this will not be your silver bullet yet. This also means streams between collections and databases are not guaranteed to be in sync. This could be an issue for someone even with a smaller number of namespaces that expect this. Please don’t get me wrong: it’s a step in the correct direction, but it falls short.

I had very high hopes for this to simplify backups in MongoDB. Percona Labs’s GitHub has a tool called MongoDB-Consistent-Backup, which tails multiple oplogs to get a consistent sharded backup without the need to pay for MongoDB’s backup service or use the complicated design that is Ops Manager (when you host it yourself). Due to the inability to do a system-wide change stream, this type of tool still needs to use the oplog. If you are not using

w:majority
  it could trigger a failure if you have an election or if a rollback occurs. Still, I have hopes this will be something that can be considered in the future to make things better for everyone.

Retryable writes

Unlike change streams, this feature is much more helpful to the general MongoDB audience. I am very excited for it. If you have not watched this video, please do right now! Samantha does a good job explaining the issue and solution in detail. However, for now just know there has been a problem that where a write that has an issue (network, app shutdown, DB shutdown, election), you had no way to know if the write failed or not. This unknown situation was terrible for a developer, and they would not know if they needed to run the command again or not. This is especially true if you have an ordering system and you’re trying to remove stock from your inventory system. Sessions, as discussed before, allowed the client to reconnect to a broken connection and keep getting results to know what happened or didn’t. To me, this is the second best feature of 3.6. Only Security is more important to me personally.

Security improvement

In speaking of security, there is one change that the security community wanted (which I don’t think is that big of a deal). For years now, the MongoDB packaging for all OSs (and even the Percona Server for MongoDB packing) by default would limit the bindIP setting to localhost. This was to prevent unintended situations where you had a database open to the public. With 3.6 now the binaries also default to this. So, yes, it will help some. But I would (and have) argued that when you install a database from binaries or source, you are taking more ownership of its setup compared to using Docker, Yum or Apt.

The other major move forward, however, is something I have been waiting for since 2010. Seriously, I am not joking! It offers the ability to limit users to specific CIDR or IP address ranges. Please note MySQL has had this since at least 1998. I can’t recall if it’s just always been there, so let’s say two decades.

This is also known as “authenticationRestriction” and it’s an array you can put into the user document when creating a document. The manual describes it as:

The authentication restrictions the server enforces on the created user. Specifies a list of IP addresses and CIDR ranges from which the user is allowed to connect to the server or from which the server can accept users.

I can not overstate how huge this is. MongoDB Inc. did a fantastic job on it. Not only does it support the classic client address matching, it supports an array of these with matching on the server IP/host also. This means supporting multiple IP segments with a single user is very easy. By extension, I could see a future where you could even limit some actions by range – allowing dev/load test to drop collections, but production apps would not be allowed to. While they should have separate users, I regularly see clients who have one password everywhere. That extension would save them from unplanned outages due to human errors of connecting to the wrong thing.

We will have a whole blog talking about these changes, their importance and using them in practice. Yes, security is that important!

Major love for array and more in Aggregation

This one is a bit easier to summarize. Arrays and dates get some much-needed aggregation love in particular. I could list all the new operators here, but I feel it’s better served in a follow-up blog where we talk about each operator and how to get the most of it. I will say my favorite new option is the $hint. Finally, I can try to control the work plan a bit if it’s making bad decisions, which sometimes happens in any technology.

A better balancer

Like many other areas, there was a good deal that went into balancer improvements. However, there are a few key things that continue the work of 3.4’s parallel balancing improvements.

Some of it makes a good deal of sense for anyone in a support role, such as FTDC now also existing in mongos’. If you do not know what this is, basically MongoDB collects some core metrics and state data and puts it into binary files in dbPath for engineers at companies like Percona and MongoDB Inc. to review. That is not to say you can’t use this data also. However, think of it as a package of performance information if a bug happens. Other diagnostic type improvements include moveChunk, which provides data about what happened when it runs in its output. Previously you could get this data from the config.changeLog or config.actionLog collections in the config servers. Obviously, more and more people are learning MongoDB’s internals and this should be made more available to them.

Having talked about diagnostic items, let’s move more into the operations wheelhouse. The single biggest frustration about sharding and replica-sets is the sensitivity to time variations that cause odd issues, even when using ntpd. To this point, as of 3.6 there is now a logical clock in MongoDB. For the geekier in the crowd, this was implemented using a Lamport Clock (great video of them). For the less geeky, think of it as a cluster-wide clock preventing some of the typical issues related to varying system times. In fact, if you look closer at the oplog record format in 3.6 there is a new wt field for tracking this. Having done that, the team at MongoDB Inc. considered what other things were an issue. At times like elections of the config servers, meta refreshes did not try enough times and could cause a mongos to stop working or fail. Those days are gone! Now it will check three times as much, for a total of ten attempts before giving up. This makes the system much more resilient.

A final change that is still somewhat invisible to the user but helps make dropping collections more stable, is that they remove the issue MongoDB had about dropping and recreating sharded collections. Your namespaces look as they always have. Behind the scenes, however, they have UUID’s attached to them so that if foo.bar drops and gets recreated, it would be a different UUID. This allows for less-blocking drops. Additionally, it prevents confusion in a distributed system if we are talking about the current or old collection.

JSON Schema validation

Some non-developers might not know much about something called JSON Schema. It allows you to set rules on schema design more efficiently and strictly than MongoDB’s internal rules. With 3.6, you can use this directly. Read more about it here. Some key points:

  • Describes your existing data format
  • Clear, human- and machine-readable documentation
  • Complete structural validation, useful for:
    • Automated testing
    • Validating client-submitted data
You can even make it reject when certain fields are missing. As for MySQL DBAs, you might ask why this is a big deal? You could always have a DBA define a schema in an RDBMS, and the point of MongoDB was to be flexible. That’s a fair and correct view. However, the big point of using it is you could apply this in production, not in development. This gives developers the freedom to move quickly, but provides operational teams with control methods to understand when mistakes or bugs are present before production is adversely affected. Taking a step back, its all about bridging the control and freedom ravines to ensure both camps are happy and able to do their jobs efficiently.

Compass is Community

If you have never used Compass, you might think this isn’t that great. You could use things like RoboMongo and such. You absolutely could, but Compass can do visualization as well as CRUD operations. It’s also a very fluid experience that everyone should know is available for use. This is especially true for QA teams who want to review how often some types of data are present, or a business analyst who needs to understand in two seconds what your demographics are.

Major WiredTiger internal overhaul

There is so much here that I recommend any engineer-minded person take a look at this deck, presented by one of the great minds behind WiredTiger. It does a fantastic job explaining all the reasons behind some of the 3.2 and 3.4 scaling issues MongoDB had on WiredTiger. Of particular interest is why it tended to have worse and worse performance as you added more collections and indexes. It then goes into how they fixed those issues:

  • Some key points on what they did
  • Made Evictions Smarts, as they are not collection uniform
  • Improved assumption around the handle cache
  • Made Checkpoints better in all ways
  • Aggressively cleaned up old handles

I hope this provides a peek into the good, bad, and ugly in MongoDB 3.6 Community! Please check back as we publish more in-depth blogs on how these features work in practice, and how to best use them.

by David Murphy at December 06, 2017 10:09 PM

Webinar Thursday, December 7, 2017: Percona XtraDB Cluster (PXC) 101

Percona XtraDB Cluster

Percona XtraDB ClusterJoin Percona’s Software Engineer (PXC Lead), Krunal Bauskar as he presents Percona XtraDB Cluster 101 on Thursday, December 7, 2017, at 7:00 am PST / 10:00 am EST (UTC-8).

Tags: Percona XtraDB Cluster, MySQL, High Availability, Clustering

Experience Level: Beginner

Percona XtraDB Cluster (PXC) is a multi-master solution that offers virtual synchronous replication among clustering node. It is based on the Codership Galera replication library. In this session, we will explore some key features of Percona XtraDB Cluster that make it enterprise ready including some recently added 5.7 exclusive features.

This webinar is an introductory and will cover the following topics:

  • ProxySQL load balancer
  • Multi-master replication
  • Synchronous replication
  • Data at rest encryption
  • Improved SST Security through simplified configuration
  • Easy to setup encrypted between-nodes communication
  • ProxySQL-assisted Percona XtraDB Cluster maintenance mode
  • Automatic node provisioning
  • Percona XtraDB Cluster “strict-mode”

Register for the webinar now.

Percona XtraDB ClusterKrunal Bauskar, C/C++ Engineer

Krunal joined Percona in September 2015. Before joining Percona he worked as part of the InnoDB team at MySQL/Oracle. He authored most of the temporary table revamp work besides a lot of other features. In the past, he was associated with Yahoo! Labs researching on big data problems and database startup which is now part of Teradata. His interest mainly includes data-management at any scale, and he has been practicing it for more than a decade now. He loves to spend time with his family or get involved in social work, unless he is out for some near-by exploration drive. He is located out of Pune, India.

by Krunal Bauskar at December 06, 2017 03:22 PM

Percona Monitoring and Management 1.5.2 Is Now Available

Percona Monitoring and Management

Percona Monitoring and ManagementPercona announces the release of Percona Monitoring and Management 1.5.2. This release contains fixes to bugs found after Percona Monitoring and Management 1.5.1 was released.

Bug fixes

  • PMM-1790QAN displayed query metrics even for a host that was not configured for mysql:queries or mongodb:queries. We have fixed the behaviour to display an appropriate warning message when there are no query metrics for the selected host.
  • PMM-1826: If PMM Server 1.5.0 is deployed via Docker, the Update button would not upgrade the instance to a later release.
  • PMM-1830: If PMM Server 1.5.0 is deployed via AMI (Amazon Machine Image) instance, the Update button would not upgrade the instance to a later release.

by Borys Belinsky at December 06, 2017 01:27 PM

December 05, 2017

Jean-Jerome Schmidt

Deploying MySQL, MariaDB, Percona Server, MongoDB or PostgreSQL - Made Easy with ClusterControl

Helping users securely automate and manage their open source databases has been at the core of our efforts from the inception of Severalnines.

And ever since the first release of our flagship product, ClusterControl, it’s always been about making it as easy and secure as possible for users to deploy complex, open source database cluster technologies in any environment.

Since our first steps with deployment, automation and management we’ve perfected the art of securely deploying highly available open source database infrastructures by developing ClusterControl from a deployment and monitoring tool to a full-blown automation and management system adopted by thousands of users worldwide.

As a result, ClusterControl can be used today to deploy, monitor, and manage over a dozen versions of the most popular open source database technologies - on premise or in the cloud.

Whether you’re looking to deploy MySQL standalone, MySQL replication, MySQL Cluster, Galera Cluster, MariaDB, MariaDB Cluster, Percona XtraDB and Percona Server for MongoDB, MongoDB itself and PostgreSQL - ClusterControl has you covered.

In addition to the database stores, users can also deploy and manage load balancing technologies such as HAProxy, ProxySQL, MaxScale and Keepalived.

“Very easy to deploy a cluster, also it facilitates administration and monitoring.”

Michel Berger IT Applications Manager European Broadcasting Union (EBU)

Using ClusterControl, database clusters can be either deployed new or existing ones imported.

A deployment wizard makes it easy and secure to deploy production-ready database clusters with a point and click interface that walks the users through the deployment process step by step.

Select Deploy or Import Cluster

ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

Walk through of the Deploy Wizard

View your cluster list

“ClusterControl is great for deploying and managing a high availability infrastructure. Also find the interface very easy to manage.”

Paul Masterson, Infrastructure Architect, Dunnes

Deploying with the ClusterControl CLI

Users can also chose to work with our CLI, which allows for easy integration with infrastructure orchestration tools such as Ansible etc.

s9s cluster     
  --create     
  --cluster-type=galera     
  --nodes='10.10.10.26;10.10.10.27;10.10.10.28'     
  --vendor=percona         
  --cluster-name=PXC_CENTOS7
  --provider-version=5.7  
  --os-user=vagrant   --wait

The ClusterControl deployment supports multiple NICS and templated configurations.

In short, ClusterControl provides:

  • Topology-aware deployment jobs for MySQL, MariaDB,Percona, MongoDB and PostgreSQL
  • Self-service and on-demand
  • From standalone nodes to load-balanced clusters
  • Your choice of barebone servers, private/public cloud and containers

To see for yourself, download ClusterControl today and give us your feedback.

by jj at December 05, 2017 07:26 PM

Peter Zaitsev

MySQL 8.0 Window Functions: A Quick Taste

Window Functions

Window FunctionsIn this post, we’ll briefly look at window functions in MySQL 8.0.

One of the major features coming to MySQL 8.0 is the support of Window functions. The detailed documentation is already available here Window functions. I wanted to take a quick look at the cases where window functions help.

Probably one the most frequent limitations in MySQL SQL syntax was analyzing a dataset. I tried to find the answer to the following question: “Find the Top N entries for each group in a grouped result.”

To give an example, I will refer to this request on Stackoverflow. While there is a solution, it is hardly intuitive and portable.

This is a popular problem, so databases without window support try to solve it in different ways. For example, ClickHouse introduced a special extension for LIMIT. You can use LIMIT n BY m to find “m” entries per group.

This is a case where window functions come in handy.

As an example, I will take the IMDB database and find the TOP 10 movies per century (well, the previous 20th and the current 21st).To download the IMDB dataset, you need to have to have an AWS account and download data from S3 storage (the details are provided on IMDB page).

I will use the following query with MySQL 8.0.3:

SELECT primaryTitle,century*100,rating,genres,rn as `Rank` FROM (SELECT primaryTitle,startYear div 100 as century,rating,genres, RANK() OVER (PARTITION BY startYear div 100 ORDER BY rating desc) rn FROM title,ratings WHERE title.tconst=ratings.tconst AND titleType='movie' AND numVotes>100000) t1 WHERE rn<=10 ORDER BY century,rating desc

The main part of this query is RANK() OVER (PARTITION BY startYear div 100 ORDER BY rating desc), which is the mentioned window function. PARTITION BY divides rows into groups, ORDER BY specifies the order and RANK() calculates the rank using the order in the specific group.

The result is:

+---------------------------------------------------+-------------+--------+----------------------------+------+
| primaryTitle                                      | century*100 | rating | genres                     | Rank |
+---------------------------------------------------+-------------+--------+----------------------------+------+
| The Shawshank Redemption                          |        1900 |    9.3 | Crime,Drama                |    1 |
| The Godfather                                     |        1900 |    9.2 | Crime,Drama                |    2 |
| The Godfather: Part II                            |        1900 |      9 | Crime,Drama                |    3 |
| 12 Angry Men                                      |        1900 |    8.9 | Crime,Drama                |    4 |
| The Good, the Bad and the Ugly                    |        1900 |    8.9 | Western                    |    4 |
| Schindler's List                                  |        1900 |    8.9 | Biography,Drama,History    |    4 |
| Pulp Fiction                                      |        1900 |    8.9 | Crime,Drama                |    4 |
| Star Wars: Episode V - The Empire Strikes Back    |        1900 |    8.8 | Action,Adventure,Fantasy   |    8 |
| Forrest Gump                                      |        1900 |    8.8 | Comedy,Drama,Romance       |    8 |
| Fight Club                                        |        1900 |    8.8 | Drama                      |    8 |
| The Dark Knight                                   |        2000 |      9 | Action,Crime,Drama         |    1 |
| The Lord of the Rings: The Return of the King     |        2000 |    8.9 | Adventure,Drama,Fantasy    |    2 |
| The Lord of the Rings: The Fellowship of the Ring |        2000 |    8.8 | Adventure,Drama,Fantasy    |    3 |
| Inception                                         |        2000 |    8.8 | Action,Adventure,Sci-Fi    |    3 |
| The Lord of the Rings: The Two Towers             |        2000 |    8.7 | Action,Adventure,Drama     |    5 |
| City of God                                       |        2000 |    8.7 | Crime,Drama                |    5 |
| Spirited Away                                     |        2000 |    8.6 | Adventure,Animation,Family |    7 |
| Interstellar                                      |        2000 |    8.6 | Adventure,Drama,Sci-Fi     |    7 |
| The Intouchables                                  |        2000 |    8.6 | Biography,Comedy,Drama     |    7 |
| Gladiator                                         |        2000 |    8.5 | Action,Adventure,Drama     |   10 |
| Memento                                           |        2000 |    8.5 | Mystery,Thriller           |   10 |
| The Pianist                                       |        2000 |    8.5 | Biography,Drama,Music      |   10 |
| The Lives of Others                               |        2000 |    8.5 | Drama,Thriller             |   10 |
| The Departed                                      |        2000 |    8.5 | Crime,Drama,Thriller       |   10 |
| The Prestige                                      |        2000 |    8.5 | Drama,Mystery,Sci-Fi       |   10 |
| Like Stars on Earth                               |        2000 |    8.5 | Drama,Family               |   10 |
| Whiplash                                          |        2000 |    8.5 | Drama,Music                |   10 |
+---------------------------------------------------+-------------+--------+----------------------------+------+
27 rows in set (0.19 sec)

The previous century was dominated by “The Godfather” and the current one by “The Lord of the Rings”. While we may or may not agree with the results, this is what the IMDB rating tells us.
If we look at the result set, we can see that there are actually more than ten movies per century, but this is how function RANK() works. It gives the same RANK for rows with an identical rating. And if there are multiple rows with the same rating, all of them will be included in the result set.

I welcome the addition of window functions into MySQL 8.0. This definitely simplifies some complex analytical queries. Unfortunately, complex queries still will be single-threaded — this is a performance limiting factor. Hopefully, we can see multi-threaded query execution in future MySQL releases.

by Vadim Tkachenko at December 05, 2017 06:21 PM

Webinar Wednesday, December 6, 2017: Gain a MongoDB Advantage with the Percona Memory Engine

Percona Memory Engine

Percona Memory EngineJoin Percona’s, CTO, Vadim Tkachenko as he presents Gain a MongoDB Advantage with the Percona Memory Engine on Wednesday, December 6, 2017, at 11:00 am PST / 2:00 pm EST (UTC-8).

Experience: Entry Level to Intermediate

Tags: Developer, DBAs, Operations

Looking for the performance of Redis or Memcache, the expressiveness of the MongoDB query language and simple high availability and sharding? Percona Memory Engine, available as part of Percona Server for MongoDB, has it all!

In this webinar, Vadim explains the architecture of the MongoDB In-Memory storage engine. He’ll also show some benchmarks compared to disk-based storage engines and other in-memory technologies.

Vadim will share specific use cases where Percona Memory Engine for MongoDB excels, such as:

  • Caching documents
  • Highly volatile data
  • Workloads with predictable response time requirements

Register for the webinar now.

Vadim TkachenkoVadim Tkachenko, CTO

Vadim Tkachenko co-founded Percona in 2006 and serves as its Chief Technology Officer. Vadim leads Percona Labs, which focuses on technology research and performance evaluations of Percona’s and third-party products. Percona Labs designs no-gimmick tests of hardware, filesystems, storage engines, and databases that surpass the standard performance and functionality scenario benchmarks. Vadim’s expertise in LAMP performance and multi-threaded programming help optimize MySQL and InnoDB internals to take full advantage of modern hardware. Oracle Corporation and its predecessors have incorporated Vadim’s source code patches into the mainstream MySQL and InnoDB products. He also co-authored the book High-Performance MySQL: Optimization, Backups, and Replication 3rd Edition. Previously, he founded a web development company in his native Ukraine and spent two years in the High-Performance Group within the official MySQL support team. Vadim received a BS in Economics and an MS in computer science from the National Technical University of Ukraine.

 

by Vadim Tkachenko at December 05, 2017 05:06 PM

December 04, 2017

MariaDB AB

MariaDB Connector/C 2.3.4 now available

MariaDB Connector/C 2.3.4 now available dbart Mon, 12/04/2017 - 12:37

The MariaDB project is pleased to announce the immediate availability of MariaDB Connector/C 2.3.4. See the release notes and changelogs for details and visit mariadb.com/downloads/connector to download.

Download MariaDB Connector/C 2.3.4

Release Notes Changelog About MariaDB Connector/C

The MariaDB project is pleased to announce the immediate availability of MariaDB Connector/C 2.3.4. See the release notes and changelog for details.

Login or Register to post comments

by dbart at December 04, 2017 05:37 PM

Peter Zaitsev

Internal Temporary Tables in MySQL 5.7

InnoDB row operations graph from PMM

In this blog post, I investigate a case of spiking InnoDB Rows inserted in the absence of a write query, and find internal temporary tables to be the culprit.

Recently I was investigating an interesting case for a customer. We could see the regular spikes on a graph depicting “InnoDB rows inserted” metric (jumping from 1K/sec to 6K/sec), however we were not able to correlate those spikes with other activity. The

innodb_row_inserted
 graph (picture from PMM demo) looked similar to this (but on a much larger scale):

InnoDB row operations graph from PMM

Other graphs (Com_*, Handler_*) did not show any spikes like that. I’ve examined the logs (we were not able to enable general log or change the threshold of the slow log), performance_schema, triggers, stored procedures, prepared statements and even reviewed the binary logs. However, I was not able to find any single write query which could have caused the spike to 6K rows inserted.

Finally, I figured out that I was focusing on the wrong queries. I was trying to correlate the spikes on the InnoDB Rows inserted graph to the DML queries (writes). However, the spike was caused by SELECT queries! But why would SELECT queries cause the massive InnoDB insert operation? How is this even possible?

It turned out that this is related to temporary tables on disk. In MySQL 5.7 the default setting for internal_tmp_disk_storage_engine is set for InnoDB. That means that if the SELECT needs to create a temporary table on disk (e.g., for GROUP BY) it will use the InnoDB storage engine.

Is that bad? Not necessarily. Krunal Bauskar published a blog post originally about the InnoDB Intrinsic Tables performance in MySQL 5.7. The InnoDB internal temporary tables are not redo/undo logged. So in general performance is better. However, here is what we need to watch out for:

  1. Change of the place where MySQL stores temporary tables. InnoDB temporary tables are stored in ibtmp1 tablespace file. There are a number of challenges with that:
    • Location of the ibtmp1 file. By default it is located inside the innodb datadir. Originally MyISAM temporary tables were stored in  tmpdir. We can configure the size of the file, but the location is always relative to InnoDB datadir, so to move it to tmpdir we need something like this: 
      innodb_temp_data_file_path=../../../tmp/ibtmp1:12M:autoextend
    • Like other tablespaces it never shrinks back (though it is truncated on restart). The huge temporary table can fill the disk and hang MySQL (bug opened). One way to fix that is to set the maximum size of ibtmp1 file: 
      innodb_temp_data_file_path=ibtmp1:12M:autoextend:max:1G
    • Like other InnoDB tables it has all the InnoDB limitations, i.e., InnoDB row or column limits. If it exceeds these, it will return “Row size too large” or “Too many columns” errors. The workaround is to set internal_tmp_disk_storage_engine to MYISAM.
  2. When all temp tables go to InnoDB, it may increase the total engine load as well as affect other queries. For example, if originally all datasets fit into buffer_pool and temporary tables were created outside of the InnoDB, it will not affect the InnoDB memory footprint. Now, if a huge temporary table is created as an InnoDB table it will use innodb_buffer_pool and may “evict” the existing pages so that other queries may perform slower.

Conclusion

Beware of the new change in MySQL 5.7, the internal temporary tables (those that are created for selects when a temporary table is needed) are stored in InnoDB ibtmp file. In most cases this is faster. However, it can change the original behavior. If needed, you can switch the creation of internal temp tables back to MyISAM: 

set global internal_tmp_disk_storage_engine=MYISAM

by Alexander Rubin at December 04, 2017 02:51 PM

MariaDB AB

MariaDB MaxScale Setup with Binlog Server and SQL Query Routing

MariaDB MaxScale Setup with Binlog Server and SQL Query Routing massimiliano_pinto_g Mon, 12/04/2017 - 03:26

Binlog server is a MariaDB MaxScale replication proxy setup which involves one Master server and several slave servers using MariaDB replication protocol.

Up to MariaDB MaxScale version 2.1, due to the lack of some SQL variables needed by the monitor for MariaDB instances, it’s not possible to use it in conjunction with SQL routing operations, such as Read/Write split routing.

With MariaDB MaxScale 2.2 (currently in beta) this is no longer a limitation as the monitor can detect a Binlog server setup and SQL statements can be properly routed among Master and Slave servers.

Depending on the configuration value of the optional variable “master_id”, the binlog server can be seen as a ‘Relay Master’ with its own slaves or just a ‘Running’ server, without its slaves being listed.

MariaDB MaxScale configuration:

# binlog server details
[binlog_server]
type=server
address=127.0.0.1
port=8808
protocol=MySQLBackend

# Mysql monitor
[MySQL Monitor]
type=monitor
module=mysqlmon
servers=server1,server2,...,binlog_server
user=mon_user
passwd=some_pass
monitor_interval=10000
detect_replication_lag=true

# R/W split service
[Read-Write-Service]
type=service
router=readwritesplit
servers=server1,server2,...

# Binlog server configuration
[Replication_Service]
type=service
router=binlogrouter
version_string=10.1.17-log
router_options=server_id=93

# Binlog server listener
[BinlogServer_Listener]
type=listener
service=Replication_Service
protocol=MySQLClient
port=8808
address=0.0.0.0


Note: the ‘binlog_server’ is not needed in the server list of R/W split service; if set it doesn’t harm MariaDB MaxScale as it doesn’t have Slave or Master states.

Binlog Server identity post reminds which parameters affect the way MaxScale is seen from Slave servers and MaxScale monitor.


Scenario A: only server_id is given in configuration.

MySQL [(none)]> select @@server_id; // The server_id of master, query from slaves.
+-------------+
| @@server_id |
+-------------+
|       10124 |
+-------------+

MySQL [(none)]> select @@server_id, @@read_only; // Maxscale server_id, query from MySQL monitor only.

+-------------+-------------+
| @@server_id | @@read_only |
+-------------+-------------+
|          93 |           0 |
+-------------+-------------+

MySQL [(none)]> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
              Slave_IO_State: Binlog Dump
                 Master_Host: 192.168.100.11  // Master server IP
                 Master_User: repo
                 Master_Port: 3306
                 ...
            Master_Server_Id: 10124 // Master Server_ID
                 ...

MaxAdmin> show servers
Server 0x1f353b0 (server1)
    Server:                              127.0.0.1
    Status:                              Slave, Running
    Protocol:                            MySQLBackend
    Port:                                25231
    Server Version:                      10.0.21-MariaDB-log
    Node Id:                             101
    Master Id:                           10124
    Slave Ids:                           
    Repl Depth:                          1
    ...

Server 0x1f31af0 (server2)
    Server:                              192.168.122.1
    Status:                              Master, Running
    Protocol:                            MySQLBackend
    Port:                                10124
    Server Version:                      10.1.24-MariaDB
    Node Id:                             10124
    Master Id:                           -1
    Slave Ids:                           101, 93
    Repl Depth:                          0
    ...

Server 0x1f32d90 (binlog_server)
    Server:                              127.0.0.1
    Status:                              Running
    Protocol:                            MySQLBackend
    Port:                                8808
    Server Version:                      10.1.17-log
    Node Id:                             93
    Master Id:                           10124
    Slave Ids:                           
    Repl Depth:                          1
    ...

Scenario B: server_id and common_identity (master_id)

[BinlogServer]
type=service
router=binlogrouter
version_string=10.1.17-log
router_options=server-id=93, master_id=1111

MySQL [(none)]> select @@server_id; // Maxscale common identity
+-------------+
| @@server_id |
+-------------+
|        1111 |
+-------------+
1 row in set (0.00 sec)

MySQL [(none)]> select @@server_id, @@read_only; // Maxscale common identity
+-------------+-------------+
| @@server_id | @@read_only |
+-------------+-------------+
|        1111 |           0 |
+-------------+-------------+
1 row in set (0.00 sec)

MySQL [(none)]> show slave status\G
*************************** 1. row ***************************
              Slave_IO_State: Binlog Dump
                 Master_Host: 192.168.100.11  // Master server IP
                 Master_User: repl
                 Master_Port: 3306
                 ...
            Master_Server_Id: 10124 // Master Server_ID
                 ...

MaxAdmin> show servers
Server 0x24103b0 (server1)
    Server:                              127.0.0.1
    Status:                              Slave, Running
    Protocol:                            MySQLBackend
    Port:                                25231
    Server Version:                      10.0.21-MariaDB-log
    Node Id:                             101
    Master Id:                           1111
    Slave Ids:                           
    Repl Depth:                          2
    ...
Server 0x240dd90 (binlog_server)
    Server:                              127.0.0.1
    Status:                              Relay Master, Running
    Protocol:                            MySQLBackend
    Port:                                8808
    Server Version:                      10.1.17-log
    Node Id:                             1111
    Master Id:                           10124
    Slave Ids:                           101
    Repl Depth:                          1
    ...

Server 0x240caf0 (server2)
    Server:                              192.168.122.1
    Status:                              Master, Running
    Protocol:                            MySQLBackend
    Port:                                10124
    Server Version:                      10.1.24-MariaDB
    Node Id:                             10124
    Master Id:                           -1
    Slave Ids:                           1111
    Repl Depth:                          0
    ...

The latter configuration with the extra master_id option is clearly then one which well represents the setup with Binlog server as a replication proxy: the user can immediately see that.

The picture shows the setup and makes it clear MariaDB MaxScale handles both replication protocol between Master and Slaves and also routes Read and Write application traffic.

complete_setup.jpg

Conclusion

This post shows how it's easy for any user to improve a MariaDB replication setup with MariaDB MaxScale combining benefits of replication proxy and query routing scalability.

MariaDB MaxScale 2.2 is in beta and we do not recommend using it in production environments. However, we do encourage you to download, test it and share your successes!

 

Additional Resources

With MariaDB MaxScale 2.2 (currently in beta) the monitor for MariaDB instances can detect a Binlog server setup and SQL statements can be properly routed among Master and Slave servers.

Login or Register to post comments

by massimiliano_pinto_g at December 04, 2017 08:26 AM

December 02, 2017

Valeriy Kravchuk

Using strace for MySQL Troubleshooting

I'd say that strace utility is even more useful for MySQL DBAs than lsof. Basically, it is a useful general purpose diagnostic and debugging tool for tracing system calls some process makes and signals it receives. The name of each system call, its arguments and its return value are printed to stderr or to the file specified with the -o option.

In context of MySQL strace is usually used to find what files mysqld process accesses, and to check the details about any I/O errors. For example, if I'd really want to try to verify Bug #88479 - "Unable to start mysqld using a single config file (and avoiding reading defaults)" by Simon Mudd, I'd just run mysqld from some 5.7.x version as a command argument for strace. On my Ubuntu 14.04 netbook I have the following files:
openxs@ao756:~/dbs/5.7$ ls -l /usr/my.cnf
-rw-r--r-- 1 root root 943 лип 19  2013 /usr/my.cnf
openxs@ao756:~/dbs/5.7$ ls -l /etc/my.cnf
-rw-r--r-- 1 root root 260 чер 24 20:40 /etc/my.cnf
openxs@ao756:~/dbs/5.7$ ls -l /etc/mysql/my.cnf
-rw-r--r-- 1 root root 116 лют 26  2016 /etc/mysql/my.cnf
openxs@ao756:~/dbs/5.7$ bin/mysqld --version
bin/mysqld  Ver 5.7.18 for Linux on x86_64 (MySQL Community Server (GPL))
So, what happens if I try to run mysqld --defaults-file=/etc/mysql/my.cnf, like this:
openxs@ao756:~/dbs/5.7$ strace bin/mysqld --defaults-file=/etc/mysql/my.cnf --print-defaults 2>&1 | grep 'my.cnf'
stat("/etc/mysql/my.cnf", {st_mode=S_IFREG|0644, st_size=116, ...}) = 0
open("/etc/mysql/my.cnf", O_RDONLY)     = 3
openxs@ao756:~/dbs/5.7$
It seems we proved that only the file passed as --defaults-file is read (if it exists). By default other locations are also checked in a specific order (note that return codes are mapped to symbolic errors when possible):
openxs@ao756:~/dbs/5.7$ strace bin/mysqld --print-defaults --print-defaults 2>&1 | grep 'my.cnf'
stat("/etc/my.cnf", {st_mode=S_IFREG|0644, st_size=260, ...}) = 0
open("/etc/my.cnf", O_RDONLY)           = 3
stat("/etc/mysql/my.cnf", {st_mode=S_IFREG|0644, st_size=116, ...}) = 0
open("/etc/mysql/my.cnf", O_RDONLY)     = 3
stat("/home/openxs/dbs/5.7/etc/my.cnf", 0x7ffd68f0e020) = -1 ENOENT (No such file or directory)
stat("/home/openxs/.my.cnf", 0x7ffd68f0e020) = -1 ENOENT (No such file or directory)
openxs@ao756:~/dbs/5.7$
If we think that --print-defaults may matter, we can try without it:
openxs@ao756:~/dbs/5.7$ strace bin/mysqld --defaults-file=/etc/mysql/my.cnf 2>&1 | grep 'my.cnf'stat("/etc/mysql/my.cnf", {st_mode=S_IFREG|0644, st_size=116, ...}) = 0
open("/etc/mysql/my.cnf", O_RDONLY)     = 3
^C
openxs@ao756:~/dbs/5.7$
Last example also shows how one can terminate tracing with Ctrl-C.

Now, let me illustrate typical use cases with (surprise!) some public MySQL bug reports where strace was important to find or verify the bug:
  • Bug #20748 - "Configuration files should not be read more than once". In this old bug report Domas Mituzas proved the point by running mysqld as a command via strace. He filtered out lines related to opening my.cnf file with egrep and made it obvious that one file may be read more than once. The bug is closed long time ago, but it is not obvious that all possible cases are covered, based on further comments...
  • Bug #49336 - "mysqlbinlog does not accept input from stdin when stdin is a pipe". Here bug reporter shown how run mysqlbinlog as a command under strace and redirect output to the file with -o option.
  • Bug #62224 - "Setting open-files-limit above the hard limit won't be logged in errorlog". This minor bug in all versions before old 5.6.x (that still remains "Verified", probably forgotten) was reported by Daniël van Eeden. strace allowed him to show what limits are really set by setrlimit() calls. The fact that arguments of system calls are also shown matters sometimes.
  • Bug #62578 - "mysql client aborts connection on terminal resize". This bug report by Jervin Real from Percona is one of my all time favorites! I clearly remember how desperate customer tried to to dump data and load them on a remote server with a nice command line, and after waiting for many days (mysqldump was run for a data set of 5TB+, don't ask, not my idea) complained that they got "Lost connection to MySQL server during query" error message and loading failed. Surely, the command was run from the terminal window on his Mac and in the process of moving it here and there he just resized the window  by chance... You can see nice example of strace usage with MySQL Sandbox in the bug report, as well as some outputs for SIGWINCH signal. Note that the bug is NOT fixed by Oracle in MySQL 5.5.x (and never happened in 5.6+), while Percona fixed it since 5.5.31.
  • Bug #65956 - "client and libmysqlclient VIO drops connection if signal received during read()". In this yet another 5.5 only regression bug that was NOT fixed in 5.5.x by Oracle (until in frames of Bug #82019 - "Is client library supposed to retry EINTR indefinitely or not" the patch was contributed for a special case of it by Laurynas Biveinis from Percona, that in somewhat changed form allowed to, hopefully, get some fix in 5.5.52+), the problem was noted while trying to attach strace to running client program (with -p option).
  • Bug #72340 - "my_print_defaults fails to parse include directive if there is no new line". Here bug reporter used strace (with -f option to trace child/forked processes and -v option get verbose output) to show that file name is NOT properly recognized by my_print_defaults. The bug is still "Verified".
  • Bug #72701 - "mysqlbinlog uses localtime() to print events, causes kernel mutex contention". Yet another nice bug report by Domas Mituzas, who had used -e stat options of strace to trace only stat() calls and show how many times they are applied to /etc/localtime.The bug is fixed since 5.5.41, 5.6.22 and 5.7.6 removed related kernel mutex contention.
  • Bug #76627 - "MySQL/InnoDB mix buffered and direct IO". It was demonstrated with strace that InnoDB was opening each .ibd file twice (this is expected as explained by Marko Mäkelä) in different modes (and this was not any good). The bug is fixed in recent renough MySQL server versions.
  • Bug #80020 - "mysqlfrm doesn't work with 5.7". In this bug report Aleksandr Kuzminsky had to use strace to find out why mysqlfrm utility failed mostly silently even with --verbose option.
  • Bug #80319 - ""flush tables" semantics deviate from manual". Here Jörg Brühe proved with strace that close() system call is not used for individual InnoDB table t when FLUSH TABLES t is executed. manual had to be clarified.
  • Bug #80889 - "PURGE BINARY LOGS TO is reading the whole binlog file and causing MySql to Stall". By attaching strace to running mysqld process and running the command it was shown that when GTID_MODE=OFF the entire file we purge to was read. No reason to do this, really. Note how return value of open(), file descriptor, was further used to track reads from this specific file.
  • Bug #81443 - "mysqld --initialize-insecure silently fails with --user flag". As MySQL DBA, you should be ready to use strace when requested by support or developers. Here bug reporter was asked to use it, and this revealed a permission problem (that was a result of the bug). No need to guess what may go wrong with permissions - just use strace to find out!
  • Bug #84708 - "mysqld fills up error logs with [ERROR] Error in accept: Bad file descriptor". Here strace was used to find out that "When mysqld is secured with tcp_wrappers it will close a socket from an unauthorized ip address and then immediately start polling that socket"... The bug is fixed in MySQL 5.7.19+ and 8.0.2+, so take care!
  • Bug #86117 - "reconnect is very slow". I do not care about MySQL Shell, but in this bug report Daniël van Eeden used -r option of strace to print a relative timestamp for every system call, and this was it was clear how much time was spent during reconnect, and where it was spent. Very useful!
  • Bug #86462 - "mysql_ugprade: improve handling of upgrade errors". In this case strace allowed Simon Mudd to find out what exact SQL statement generated by mysql_upgrade failed.
The last but not the least, note that you may need root or sudo privileges to use strace. On my Ubuntu 14.04 this kind of messages may appear even if I am the user that owns mysqld process (same with gdb):
openxs@ao756:~/dbs/maria10.2$ strace -p 2083
strace: attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
To summarize, strace may help MySQL DBA to find out:

  • what files are accessed by the mysqld process or related utilities, and in what order
  • why some MySQL-related command (silently) fails or hangs
  • why some commands end up with permission denied or other errors
  • what signals MySQL server and tools get
  • what system calls could took a lot of time when something works slow
  • when files are opened and closed, and how much data are read from the files
  • where the error log and other logs are really located (we can look for system calls related to writing to stderr, for example)
  • how MySQL really works with files, ports and sockets
It also helps to find and verify MySQL bugs, and clarify missing details in MySQL manual.

There other similar tools for tracing system calls (maybe among other things) on Linux that I am going to review in this blog some day. Performance impact of running MySQL server under this kind of tracing is also a topic to study.

by Valeriy Kravchuk (noreply@blogger.com) at December 02, 2017 04:42 PM

December 01, 2017

Peter Zaitsev

Percona Monitoring and Management 1.5: QAN in Grafana Interface

Percona-Monitoring-and-Management-1.5-QAN-1 small

In this post, we’ll examine how we’ve improved the GUI layout for Percona Monitoring and Management 1.5 by moving the Query Analytics (QAN) functions into the Grafana interface.

For Percona Monitoring and Management users, you might notice that QAN appears a little differently in our 1.5 release. We’ve taken steps to unify the PMM interface so that it feels more natural to move from reviewing historical trends in Metrics Monitor to examining slow queries in QAN.  Most significantly:

  1. QAN moves from a stand-alone application into Metrics Monitor as a dashboard application
  2. We updated the color scheme of QAN to match Metrics Monitor (but you can toggle a button if you prefer to still see QAN in white!)
  3. Date picker and host selector now use the same methods as Metrics Monitor

Percona Monitoring and Management 1.5 QAN 1

Starting from the PMM landing page, you still see two buttons – one for Metrics Monitor and another for Query Analytics (this hasn’t changed):

Percona Monitoring and Management 1.5 QAN 2

Once you select Query Analytics on the left, you see the new Metrics Monitor dashboard page for PMM Query Analytics. It is now hosted as a Metrics Monitor dashboard, and notice the URL is no longer /qan:

Percona Monitoring and Management 1.5 QAN 3

Another advantage of the Metrics Monitor dashboard integration is that the QAN inherits the host selector from Grafana, which supports partial string matching. This makes it simpler to find the host you’re searching for if you have more than a handful of instances:

Percona Monitoring and Management 1.5 QAN 4

The last feature enhancement worth mentioning is the native Grafana time selector, which lets you select down to the minute resolution time frames. This was a frequent source of feature requests — previously PMM limited you to our pre-defined default ranges. Keep in mind that QAN has an internal archiving job that caps QAN history at eight days.

Percona Monitoring and Management 1.5 QAN 5

Last but not least is the ability to toggle between the default dark interface and the optional white. Look for the small lightbulb icon at the bottom left of any QAN screen (Percona Monitoring and Management 1.5 QAN 6) and give it a try!

Percona Monitoring and Management 1.5 QAN 7

We hope you enjoy the new interface, and we look forward to your feedback on these improvements!

by Michael Coburn at December 01, 2017 09:21 PM

This Week in Data with Colin Charles 17: AWS Re:Invent, a New Book on MySQL Cluster and Another Call Out for Percona Live 2018

Colin Charles

Colin Charles Open Source Database evangelist for PerconaJoin Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

The CFP for Percona Live Santa Clara 2018 closes December 22, 2017: please consider submitting as soon as possible. We want to make an early announcement of talks, so we’ll definitely do a first pass even before the CFP date closes. Keep in mind the expanded view of what we are after: it’s more than just MySQL and MongoDB. And don’t forget that with one day less, there will be intense competition to fit all the content in.

A new book on MySQL Cluster is out: Pro MySQL NDB Cluster by Jesper Wisborg Krogh and Mikiya Okuno. At 690 pages, it is a weighty tome, and something I fully plan on reading, considering I haven’t played with NDBCLUSTER for quite some time.

Did you know that since MySQL 5.7.17, connection control plugins are included? They help DBAs introduce an increasing delay in server response to clients after a certain number of consecutive failed connection attempts. Read more at the connection control plugins.

While there are a tonne of announcements coming out from the Amazon re:Invent 2017 event, I highly recommend also reading Some data of interest as AWS reinvent 2017 ramps up by James Governor. Telemetry data from sumologic’s 1,500 largest customers suggest that NoSQL database usage has overtaken relational database workloads! Read The State of Modern Applications in the Cloud. Page 8 tells us that MySQL is the #1 database on AWS (I don’t see MariaDB Server being mentioned which is odd; did they lump it in together?), and MySQL, Redis & MongoDB account for 40% of database adoption on AWS. In other news, Andy Jassy also mentions that less than 1.5 months after hitting 40,000 database migrations, they’ve gone past 45,000 over the Thanksgiving holiday last week. Have you started using AWS Database Migration Service?

Releases

Link List

Upcoming appearances

  • ACMUG 2017 gathering – Beijing, China, December 9-10 2017 – it was very exciting being there in 2016, I can only imagine it’s going to be bigger and better in 2017, since it is now two days long!

Feedback

I look forward to feedback/tips via e-mail at colin.charles@percona.com or on Twitter @bytebot.

by Colin Charles at December 01, 2017 02:58 PM

Jean-Jerome Schmidt

Upgrading to the ClusterControl Enterprise Edition

The ClusterControl Enterprise Edition provides you will a full suite of management and scaling features in addition to the deployment and monitoring functions offered as part of the free Community Edition. You also have the ability to deploy, configure and manage the top open source load balancing and caching technologies to drive peak performance for your mission-critical applications.

Whether you have been benefiting from the free resources included in the Community Edition or have evaluated the product through the Enterprise Trial, we’ll walk you through how our licensing works and explain how to get you up-and-running with all the automation and scaling that ClusterControl Enterprise has to offer.

“With quick installation, ease of use, great support, stable deployments and a scalable architecture, ClusterControl is just the solution we were looking for to provide a strong MySQL HA platform to our customers.”

Xavi Morrus, CMO, MediaCloud

How to Upgrade from Community to Enterprise

While using the ClusterControl Community Edition you may have clicked on a feature and got a pop-up indicating that it was not included in the version you are using. When this happens you have two options. You can activate (or extend) your Enterprise Trial OR you can contact sales to purchase an enterprise license.

“Our back-end is reliant on different databases to tackle different tasks. Using several different tools, rather than a one-stop shop, was detrimental to our productivity. Severalnines is that ‘shop’ and we haven’t looked back. ClusterControl is an awesome solution like no other.”

Zeger Knops, Head of Business Technology, vidaXL

Enterprise Trial

The ClusterControl Enterprise trial provides you with free access to our full suite of features for 30 days. The purpose of this trial is to allow you to “kick the tires” using your environments and applications to make sure that ClusterControl meets your needs.

With the trial you have access to all our Community features plus: Custom Dashboards, Load Balancers, Configuration Management, Backup and Restore, Automatic Node and Cluster Recovery, Role Based Access Control, Key Management, LDAP, SSL Encryption Scaling, and more!

The trial also grants you Enterprise Level access to our support teams 24/7. We want to make sure that you have the best experience during your trial and also introduce you to our amazing support that you can count on when you become a customer of Severalnines.

At the end of your trial, you will have the option to meet with our sales team to continue with ClusterControl Enterprise on a paid license. Or you may also continue with our ClusterControl Community Edition, which you can use for free - forever.

Extending Your Trial

Sometimes thirty days isn’t enough time to evaluate a product as extensive as ClusterControl. In these situations we can sometimes grant an extension to allow you some more time to evaluate the product. This extension can be requested from the product itself and you will be contacted by an account manager to arrange for the extension.

“ClusterControl is phenomenal software…I’m usually not impressed with vendors or the software we buy, because usually it’s over promised and under delivered. ClusterControl is a nice handy system that makes
me feel confident that we can run this in a production environment.”

Jordan Marshall, Manager of Database Administration, Black Hills Corporation

Purchasing a Commercial License

ClusterControl offers three separate plans and different support options. Our account managers are available to assist, and recommend the best plan. We also offer volume discounts for larger orders. In short, we will work very hard to make sure our price meets your needs and budget. Once we’ve all signed on the dotted line, you will then be provided with Commercial License keys that you can put into your already deployed environment (or into a new one) which will then immediately grant you full access to the entire suite of ClusterControl features that you have contracted.

ClusterControl
Single Console for Your Entire Database Infrastructure
Find out what else is new in ClusterControl

Benefits of Upgrading

While the free ClusterControl community version provides rich features that allow you to easily and securely deploy and monitor your open source databases, the Enterprise Edition provides much much more!

These are just some of the features awaiting you in the Enterprise Edition...

  • Advanced Backup & Restoration: With ClusterControl you can schedule logical or physical backups with failover handling and easily restore backups to bootstrap nodes or systems.
  • Automated Failover: ClusterControl includes advanced support for failure detection and handling; it also allows you to deploy different proxies to integrate them with your HA stack.
  • Topology Changes: Making topology changes with ClusterControl is easy; it does all the background work to elect a new master, deploy fail-over slave servers, rebuild slaves in case of data corruption, and maintain load balancer configurations to reflect all the changes.
  • Load Balancing: Load balancers are an essential component in database high availability; especially when making topology changes transparent to applications and implementing read-write split functionality and ClusterControl provides support for ProxySQL, HAProxy, and Maxscale.
  • Advanced Security: ClusterControl removes human error and provides access to a suite of security features automatically protecting your databases from hacks and other threats. Operational Reports come in handy, whether you need to show you are meeting your SLAs or wish to keep track of the historical data of your cluster.
  • Scaling: Easily add and remove nodes, resize instances, and clone your production clusters with ClusterControl.

In short, ClusterControl is an all-inclusive database management system that removes the need for your team to have to cobble together multiple tools, saving you time and money.

If you ever have any issues during this process you can always consult the documentation or contact us. If you need support you can contact us here.

by Severalnines at December 01, 2017 02:11 PM

MariaDB AB

The Binlog Server Common Identity

The Binlog Server Common Identity massimiliano_pinto_g Fri, 12/01/2017 - 03:19

The “identity” of the binlog server layer from the slave server’s point of view is something that could be modified in order to provide all slaves such common parameters for all MariaDB MaxScale server they could replicate from.

This way the slave servers can see the same values for such config options and they will not be able to understand whether the real master has changed after a failure or a new server has been promoted as master, for any reason.

Set of parameters useful for binlog server identity configuration.

    Some parameters must  be configured with different values in each MariaDB MaxScale server:

  • server-id
    As with uuid, MariaDB MaxScale must have a unique server-id for the connection it makes to the master, this parameter provides the value of server-id that MariaDB MaxScale will use when connecting to the master.

  • uuid
    This is used to set the unique uuid that the binlog router uses when it connects to the master server.
    If no explicit value is given for the uuid in the configuration file then a uuid will be generated.

    Let’s see in the details the parameters that could be set for binlog server common identity:

  • master-id
    The server-id value that MariaDB MaxScale should use to report to the slaves that connect to MariaDB MaxScale.
    This may either be the same as the server-id of the real master or can be chosen to be different if the slaves need to be aware of the proxy layer.
    The real master server-id will be used if the option is not set.

  • master_uuid
    It is a requirement of replication that each slave have a unique UUID value. The MariaDB MaxScale router will identify itself to the slaves using the uuid of the real master if this option is not set.

  • master_version
    The MariaDB MaxScale router will identify itself to the slaves using the server version of the real master if this option is not set.

  • master_hostname
    The MariaDB MaxScale router will identify itself to the slaves using the server hostname of the real master if this option is not set.
     
  • slave_hostname
    MariaDB MaxScale can optionally identify itself to the master using a custom hostname.
    The specified hostname can be seen in the master server via SHOW SLAVE HOSTS command.
    The default is not to send any hostname string during registration.
    This parameter doesn’t affect the identity seen by the slave servers but it’s useful in order to properly list all MaxScale servers that are replicating from the master.

An example with one Master, two MariaDB MaxScale servers and many slaves per each proxy:


2_routers.jpg


Master:

  • server_id=1, server_uuid: A, version: 5.6.15

MariaDB MaxScale

  • Max_1: server-id=2, uuid=B        
  • Max_2: server-id=3, uuid=C

Slaves (1 ... N)

  • Slave_1: server_id=10,server_uuid=FFF,Ver: 5.6.18
  • Slave_2: server_id=11,server_uuid=DDD,Ver: 5.6.19
  • Slave_3: server_id=12,server_uuid=ABC,Ver: 5.6.17


The MariaDB MaxScale common identity we want to show all slaves is:

  1. master_id: 1111
  2. master_version: 5.6.99-common
  3. master_uuid: xxx-fff-cccc-fff
  4. master_hostname=common_server

 

    Detailed options set in maxscale.cnf for each MaxScale server:


    Max1

    router_options=server-id=2,
                   uuid=BBB,
                   master_version=5.6.99-common,
                   master_hostname=common_server,
                   master_uuid=xxx-fff-cccc-fff,
                   master_id=1111

    Max2

    router_options=server-id=3,
                   uuid=CCC,
                   master_version=5.6.99-common,
                   master_hostname=common_server,
                   master_uuid=xxx-fff-cccc-fff,
                   master_id=1111

    Query results for the common parameters, issued to any MaxScale server on its client port:

    MariaDB> select @@server_id;
    +-------------+
    | @@server_id |
    +-------------+
    |        1111 |
    +-------------+
    1 row in set (0.00 sec)
    
    MariaDB> select @@server_uuid;
    +------------------+
    | @@server_uuid    |
    +------------------+
    | xxx-fff-cccc-fff |
    +------------------+
    1 row in set (0.00 sec)
    
    MariaDB> select version();
    +---------------+
    | VERSION()     |
    +---------------+
    | 5.6.29-common |
    +---------------+
    1 row in set (0.00 sec)
    
    MariaDB> select @@hostname;
    +---------------+
    | @@hostname    |
    +---------------+
    | common_server |
    +---------------+
    1 row in set (0.00 sec)

    Example in the log file of Max1: both master and slave identity is logged.

    2016-01-13 11:06:45  notice : BinlogServer: identity seen by the master: server_id: 2, uuid: XYZ, Host: binlog-server-1
    2016-01-13 11:06:45  notice : BinlogServer: identity seen by the slaves: server_id: 1111, uuid: XYZ, hostname: common_server, MySQL version: 5.6.17-mxs

     

    Conclusion

    We encourage you to download MariaDB MaxScale, test the Binlog Server setup with the Common Identity variables and share your experience!

    We are done for now but stay tuned, a follow up blog post will show how to configure MariaDB MaxScale for both replication proxy and query routing applications.


    Additional Resources

    The “identity” of the binlog server layer from the slave server’s point of view is something that could be modified in order to provide all slaves such common parameters for all MariaDB MaxScale server they could replicate from.

    Login or Register to post comments

    by massimiliano_pinto_g at December 01, 2017 08:19 AM

    November 30, 2017

    MariaDB AB

    Validating a MariaDB ColumnStore System Setup

    Validating a MariaDB ColumnStore System Setup davidhill2 Thu, 11/30/2017 - 17:41

     

    We have learned that the MariaDB ColumnStore prerequisites and install can be complex for multi-node installs so we decided it would help our users to develop a tool to validate for install readiness.

    This tool will validate the setup whether installing on a single server or a multi-server system. It is called MariaDB ColumnStore Cluster Test Tool. It is part of the MariaDB ColumnStore package and can be run from the installing server once the MariaDB ColumnStore package has been installed.

    You can also use this tool on a MariaDB ColumnStore system if it fails to startup or a server has been replaced within an existing system. Some OS settings could have been changed by a System Admin or lost during a reboot. This tool will help detect those issues.

    The tool will:

    • Communicate with all the servers that are going to be used in the MariaDB ColumnStore system to test out the SSH connectivity

    • Check for matching OSs and locale settings on all servers

    • Check that each of the required dependency packages are installed on each node

    • Check the firewall settings and test the ports that the MariaDB ColumnStore product will utilize to communicate between the servers

    • Compare the system’s date and time to make sure they are in sync between the local servers and the other servers

    Running this tool will detect and report any issues that might prevent the MariaDB ColumnStore product from installing and starting up, which can save a lot of time during the initial install.

    Here is an example where the tool detected two issues on a three-node system. The two non-local nodes are referenced by the provided IP addresses in the command. This command is run from the server and is designated as ‘pm1’

     

    # ./columnstoreClusterTester.sh --ipaddr=172.30.0.59,172.30.0.152
    
    *** This is the MariaDB Columnstore Cluster System test tool ***
    
    ** Validate local OS is supported
    
    Local Node OS System Name : CentOS Linux 7 (Core)
    
    ** Run Ping access Test to remote nodes
    
    172.30.0.59  Node Passed ping test
    172.30.0.152  Node Passed ping test
    
    ** Run SSH Login access Test to remote nodes
    
    172.30.0.59  Node Passed SSH login test using ssh-keys
    172.30.0.152  Node Passed SSH login test using ssh-keys
    
    ** Run OS check - OS version needs to be the same on all nodes
    
    Local Node OS Version : CentOS Linux 7 (Core)
    
    172.30.0.59 Node OS Version : CentOS Linux 7 (Core)
    172.30.0.152 Node OS Version : CentOS Linux 7 (Core)
    
    ** Run Locale check - Locale needs to be the same on all nodes
    
    Local Node Locale : LANG=en_US.UTF-8
    172.30.0.59 Node Locale : LANG=en_US.UTF-8
    172.30.0.152 Node Locale : LANG=en_US.UTF-8
    
    ** Run SELINUX check - Setting should to be disabled on all nodes
    
    Local Node SELINUX setting is Not Enabled
    172.30.0.59 Node SELINUX setting is Not Enabled
    172.30.0.152 Node SELINUX setting is Not Enabled
    
    ** Run Firewall Services check - Firewall Services should to be Inactive on all nodes
    
    Local Node iptables service is Not Active
    Local Node ufw service is Not Active
    Local Node firewalld service is Not Active
    Local Node firewall service is Not Active
    
    172.30.0.59 Node iptables service is Not Enabled
    172.30.0.59 Node ufw service is Not Enabled
    172.30.0.59 Node firewalld service is Not Enabled
    172.30.0.59 Node firewall service is Not Enabled
    
    172.30.0.152 Node iptables service is Not Enabled
    172.30.0.152 Node ufw service is Not Enabled
    172.30.0.152 Node firewalld service is Not Enabled
    172.30.0.152 Node firewall service is Not Enabled
    
    
    ** Run MariaDB ColumnStore Port (8600-8620) availability test
    
    172.30.0.59  Node Passed port test
    172.30.0.152  Node Passed port test
    
    ** Run Date/Time check - Date/Time should be within 10 seconds on all nodes
    
    Passed: 172.30.0.59 Node date/time is within 10 seconds of local node
    Passed: 172.30.0.152 Node date/time is within 10 seconds of local node
    
    ** Run MariaDB ColumnStore Dependent Package Check
    
    Local Node - Passed, all dependency packages are installed
    
    172.30.0.59 Node - Passed, all dependency packages are installed
    
    Failed, 172.30.0.152 Node package expect is not installed, please install
    
    Failure occurred, do you want to continue? (y,n) > y
    
    
    *** Finished Validation of the Cluster, Failures occurred. Check for Error/Failed test results ***
    

     

    -----------------------------------------------------------------------------------------------------------------------

    As you can see from the report above, one issue was detected:

    • Package ‘expect’ was missing on one of the other servers.

    So we fix the one issue as shown here

    INSTALL THE MISSING PACKAGE

    # ssh 172.30.0.152 yum install expect -y
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirror.hostduplex.com
     * extras: mirrors.kernel.org
     * updates: mirrors.cat.pdx.edu
    Resolving Dependencies
    --> Running transaction check
    ---> Package expect.x86_64 0:5.45-14.el7_1 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package          Arch             Version                 Repository      Size
    ================================================================================
    Installing:
     expect           x86_64           5.45-14.el7_1           base           262 k
    
    Transaction Summary
    ================================================================================
    Install  1 Package
    
    Total download size: 262 k
    Installed size: 566 k
    Downloading packages:
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : expect-5.45-14.el7_1.x86_64                                  1/1 
      Verifying  : expect-5.45-14.el7_1.x86_64                                  1/1 
    
    Installed:
      expect.x86_64 0:5.45-14.el7_1                                                 
    
    Complete!
    

     

    -----------------------------------------------------------------------------------------------------------------------

     

    Now rerun the test to validate a clean setup

    # ./columnstoreClusterTester.sh --ipaddr=172.30.0.59,172.30.0.152
    
    *** This is the MariaDB Columnstore Cluster System test tool ***
    
    ** Validate local OS is supported
    
    Local Node OS System Name : CentOS Linux 7 (Core)
    
    ** Run Ping access Test to remote nodes
    
    172.30.0.59  Node Passed ping test
    172.30.0.152  Node Passed ping test
    
    ** Run SSH Login access Test to remote nodes
    
    172.30.0.59  Node Passed SSH login test using ssh-keys
    172.30.0.152  Node Passed SSH login test using ssh-keys
    
    ** Run OS check - OS version needs to be the same on all nodes
    
    Local Node OS Version : CentOS Linux 7 (Core)
    
    172.30.0.59 Node OS Version : CentOS Linux 7 (Core)
    172.30.0.152 Node OS Version : CentOS Linux 7 (Core)
    
    ** Run Locale check - Locale needs to be the same on all nodes
    
    Local Node Locale : LANG=en_US.UTF-8
    172.30.0.59 Node Locale : LANG=en_US.UTF-8
    172.30.0.152 Node Locale : LANG=en_US.UTF-8
    
    ** Run SELINUX check - Setting should to be disabled on all nodes
    
    Local Node SELINUX setting is Not Enabled
    172.30.0.59 Node SELINUX setting is Not Enabled
    172.30.0.152 Node SELINUX setting is Not Enabled
    
    ** Run Firewall Services check - Firewall Services should to be Inactive on all nodes
    
    Local Node iptables service is Not Active
    Local Node ufw service is Not Active
    Local Node firewalld service is Not Active
    Local Node firewall service is Not Active
    
    172.30.0.59 Node iptables service is Not Enabled
    172.30.0.59 Node ufw service is Not Enabled
    172.30.0.59 Node firewalld service is Not Enabled
    172.30.0.59 Node firewall service is Not Enabled
    
    172.30.0.152 Node iptables service is Not Enabled
    172.30.0.152 Node ufw service is Not Enabled
    172.30.0.152 Node firewalld service is Not Enabled
    172.30.0.152 Node firewall service is Not Enabled
    
    
    ** Run MariaDB ColumnStore Port (8600-8620) availability test
    
    172.30.0.59  Node Passed port test
    172.30.0.152  Node Passed port test
    
    ** Run Date/Time check - Date/Time should be within 10 seconds on all nodes
    
    Passed: 172.30.0.59 Node date/time is within 10 seconds of local node
    Passed: 172.30.0.152 Node date/time is within 10 seconds of local node
    
    ** Run MariaDB ColumnStore Dependent Package Check
    
    Local Node - Passed, all dependency packages are installed
    
    172.30.0.59 Node - Passed, all dependency packages are installed
    
    172.30.0.152 Node - Passed, all dependency packages are installed
    
    
    *** Finished Validation of the Cluster, all Test Passed ***

     

    -----------------------------------------------------------------------------------------------------------------------

     

    Please find more details about the MariaDB ColumnStore Cluster Test Tool and how to use it.

    We are excited to offer this new tool for MariaDB ColumnStore 1.1, which is available for download as part of MariaDB AX, an enterprise open source solution for modern data analytics and data warehousing.

    We have learned that the MariaDB ColumnStore prerequisites and install can be complex for multi-node installs so we decided it would help our users to develop a tool to validate for install readiness. This tool will validate the setup whether installing on a single server or a multi-server system. It is called MariaDB ColumnStore Cluster Test Tool. It is part of the MariaDB ColumnStore package and can be run from the installing server once the MariaDB ColumnStore package has been installed.

    Login or Register to post comments

    by davidhill2 at November 30, 2017 10:41 PM

    Peter Zaitsev

    Percona Server for MongoDB 3.4.10-2.10 Is Now Available

    Percona Server for MongoDB 3.4

    Percona Server for MongoDB 3.4Percona announces the release of Percona Server for MongoDB 3.4.10-2.10 on November 30, 2017. Download the latest version from the Percona web site or the Percona Software Repositories.

    Percona Server for MongoDB is an enhanced, open source, fully compatible, highly-scalable, zero-maintenance downtime database supporting the MongoDB v3.4 protocol and drivers. It extends MongoDB with Percona Memory Engine and MongoRocks storage engine, as well as several enterprise-grade features:

    Percona Server for MongoDB requires no changes to MongoDB applications or code.

    This release is based on MongoDB 3.4.10 and includes the following additional change:

    • oplog searches have been optimized in MongoRocks, which should also increase overall performance.

    by Hrvoje Matijakovic at November 30, 2017 09:50 PM

    November 29, 2017

    Peter Zaitsev

    Percona Monitoring and Management 1.5.1 Is Now Available

    Percona Monitoring and Management

    Percona announces the release of Percona Monitoring and Management 1.5.1. This release contains fixes for bugs found after Percona Monitoring and Management 1.5.0 was released.

    Bug fixes

    • PMM-1771: When upgrading PMM to 1.5.0 using Docker commands, PMM System SummaryPMM Add InstancePMM Query Analytics dashboards were not available.
    • PMM-1761: The PMM Query Analytics dashboard did not display the list of hosts correctly.
    • PMM-1769: It was possible to add an Amazon RDS instance providing invalid credentials on the PMM Add Instance dashboard.

    Other bug fixes: PMM-1767PMM-1762

    by Borys Belinsky at November 29, 2017 07:12 PM

    Jean-Jerome Schmidt

    Free Open Source Database Deployment & Monitoring with ClusterControl Community Edition

    The ClusterControl Community Edition is a free-to-use, all-in-one database management system that allows you to easily deploy and monitor the top open source database technologies like MySQL, MariaDB, Percona, MongoDB, PostgreSQL, Galera Cluster and more. It also allows you to import and monitor your existing database stack.

    Free Database Deployment

    The ClusterControl Community Edition ensures your team can easily and securely deploy production-ready open source database stacks that are built using battle-tested, proven methodologies. You don’t have to be a database expert to utilize the ClusterControl Community Edition - deploying the most popular open sources databases is easy with our point-and-click interface. Even if you are a master of deploying databases, ClusterControl’s point-and-click deployments will save you time and ensure your databases are deployed correctly, removing the chance for human error. There is also a CLI for those who prefer the command line, or need to integrate with automation scripts.

    The ClusterControl Community Edition is not restricted to a single database technology and supports the major flavors and versions. With it you’re able to apply point-and-click deployments of MySQL standalone, MySQL replication, MySQL Cluster, Galera Cluster, MariaDB, MariaDB Cluster, Percona XtraDB and Percona Server for MongoDB, MongoDB itself and PostgreSQL!

    Free Database Monitoring

    The ClusterControl Community Edition makes monitoring easy by providing you the ability to look at all your database instances across multiple data centers or drill into individual nodes and queries to pinpoint issues. Offering a high-level, multi-dc view as well as a deep-dive view, ClusterControl lets you keep track of your databases so you can keep them running at peak performance.

    In addition to monitoring the overall stack and node performance you can also monitor the specific queries to identify potential errors that could affect performance and uptime.

    Why pay for a monitoring tool when the ClusterControl Community Edition gives you a great one for free!

    Free Database Developer Studio

    The Developer Studio provides you a set of monitoring and performance advisors to use and lets you create custom advisors to add security and stability to your database infrastructures. It lets you extend the functionality of ClusterControl, which helps you detect and solve unique problems in your environments.

    We even encourage our users to share the advisors they have created on GitHub by adding a fork to our current advisor bundle. If we like them and think that they might be good for other users we’ll include them in future ClusterControl releases.

    ClusterControl
    Single Console for Your Entire Database Infrastructure
    Find out what else is new in ClusterControl

    Why Should I Use the ClusterControl Community Edition?

    These are just a few of the reasons why you should use ClusterControl as your system to deploy and monitor your open source database environments…

    • You can deploy knowing you are using proven methodologies and industry best practices.
    • If you are just getting started with open source database technology ClusterControl makes it easy for the beginner to deploy and monitor your stacks removing human error and saving you time.
    • If you are not familiar with orchestration programs like Puppet and Chef? Don’t worry! The ClusterControl Community Edition uses a point-and-click GUI to make it easy to get your environment production-ready.
    • The ClusterControl Community Edition gives you deployment and monitoring in one battle-tested all-in-one system. Why use one tool for scripting only to use a different tool for monitoring?
    • If you are not sure what database technology is right for your application? The ClusterControl Community Edition supports nearly two dozen database versions that you can try.
    • Have a load balancer running on an existing stack? With the ClusterControl Community Edition you can import and deploy your existing and already configured load balancer to run alongside your database instances.

    If you are ready to give it a try click here to download and install the latest version of ClusterControl. Each install comes with the option to activate a 30-day enterprise trial as well.

    by jj at November 29, 2017 12:36 PM

    MariaDB Foundation

    MariaDB 10.2.11 now available

    The MariaDB project is pleased to announce the availability of MariaDB 10.2.11. See the release notes and changelogs for details. Download MariaDB 10.2.11 Release Notes Changelog What is MariaDB 10.2? MariaDB APT and YUM Repository Configuration Generator Thanks, and enjoy MariaDB!

    The post MariaDB 10.2.11 now available appeared first on MariaDB.org.

    by Ian Gilfillan at November 29, 2017 05:36 AM

    November 28, 2017

    Peter Zaitsev

    Best Practices for Percona XtraDB Cluster on AWS

    Percona XtraDB Cluster on AWS 2 small

    In this blog post I’ll look at the performance of Percona XtraDB Cluster on AWS using different service instances, and recommend some best practices for maximizing performance.

    You can use Percona XtraDB Cluster in AWS environments. We often get questions about how best to deploy it, and how to optimize both performance and spend when doing so. I decided to look into it with some benchmark testing.

    For these benchmark tests, I used the following configuration:

    • Region:
      • Availability zones: US East – 1, zones: b, c, d
      • Sysbench 1.0.8
      • ProxySQL 1.4.3
      • 10 tables, 40mln records – ~95GB dataset
      • Percona XtraDB Cluster 5.7.18
      • Amazon Linux AMI

    We evaluated different AWS instances to provide the best recommendation to run Percona XtraDB Cluster. We used instances

    • With General Purpose storage volumes, 200GB each
    • With IO provisioned volumes, 200GB, 10000 IOS
    • I3 instances with local attached NVMe storage.

    We also used different instance sizes:

    Instance vCPU Memory
    r4.large 2 15.25
    r4.xlarge 4 30.5
    r4.2xlarge 8 61
    r4.4xlarge 16 122
    i3.large 2 15.25
    i3.xlarge 4 30.5
    i3.2xlarge 8 61
    i3.4xlarge 16 122

     

    While I3 instances with NVMe storage do not provide the same functionality for handling shared storage and snapshots as General Purpose and IO provisioned volumes, since Percona XtraDB Cluster provides data duplication by itself we think it is still valid to include them in this comparison.

    We ran benchmarks in the US East 1 (N. Virginia) Region, and we used different availability zones for each of the Percona XtraDB Cluster zones (mostly zones “b”, “c” and “d”):

    Percona XtraDB Cluster on AWS 1

    The client was directly connected and used ProxySQL, so we were able to measure ProxySQL’s performance overhead as well.

    ProxySQL is an advanced method to access Percona XtraDB Cluster. It can perform a health check of the nodes and route the traffic to the ONLINE node. It can also split read and write traffic and route read traffic to different nodes (although we didn’t use this capability in our benchmark).

    In our benchmarks, we used 1,4, 16, 64 and 256 user threads. For this detailed review, however, we’ll look at the 64 thread case. 

    Results

    First, let’s review the average throughput (higher is better) and latency (lower is better) results (we measured 99% percentile with one-second resolution):

    Percona XtraDB Cluster on AWS 2

    Results summary, raw performance:

    The performance for Percona XtraDB Cluster running on GP2 volumes is often pretty slow, so it is hard to recommend this volume type for the serious workloads.

    IO provisioned volumes perform much better, and should be considered as the primary target for Percona XtraDB Cluster deployments. I3 instances show even better performance.

    I3 instances use locally attached volumes and do not provide equal functionality as EBS IO provisioned volumes — although some of these limitations are covered by Percona XtraDB Cluster’s ability to keep copies of data on each node.

    Results summary for jitter:

    Along with average throughput and latency, it is important to take into account “jitter” — how stable is the performance during the runs?

    Percona XtraDB Cluster on AWS 3

    Latency variation for GP2 volumes is significant — practically not acceptable for serious usage. Let’s review the latency for only IO provisioning and NVMe volumes. The following chart provides better scale for just these two:

    Percona XtraDB Cluster on AWS 4

    At this scale, we see that NVMe provides a 99% better response time and is more stable. There is still variation for IO provisioned volumes.

    Results summary, cost

    When speaking about instance and volume types, it would be impractical to avoid mentioning of the instance costs. We need to analyze how much we need to pay to achieve the better performance. So we prepared data how much does it cost to produce throughput of 1000 transactions per second.

    We compare on-demand and reserved instances pricing (reserved for one year / all upfront / tenancy-default):

    Percona XtraDB Cluster on AWS 5

    Because IO provisioned instances give much better performance, the price performance is comparable if not better than GP2 instances.

    I3 instances are a clear winner.

    It is also interesting to compare the raw cost of benchmarked instances:

    Percona XtraDB Cluster on AWS 6

    We can see that IO provisioned instances are the most expensive, and using reserved instances does not provide much savings. To understand the reason for this, let’s take a look at how cost is calculated for components:

    Percona XtraDB Cluster on AWS 7

    So for IO provisioned volumes, the majority of the cost comes from IO provisioning (which is the same for both on-demand and reserved instances).

    Percona XtraDB Cluster scalability

    Another interesting effort is looking at how Percona XtraDB Cluster performance scales with the instance size. As we double resources (both CPU and Memory) while increasing the instance size, how does it affect Percona XtraDB Cluster?

    So let’s take a look at throughput:

    Percona XtraDB Cluster on AWS 8

    Throughput improves with increasing the instance size. Let’s calculate speedup with increasing instance size for IO provisioned and I3 instances:

    Speedup X Times to Large Instance IO1 i3
    large 1 1
    xlarge 2.67 2.11
    2xlarge 5.38 4.31
    4xlarge 5.96 7.83

     

    Percona XtraDB Cluster can scale (improve performance) with increasing instance size. Keep in mind, however, that it depends significantly on the workload. You may not get the same performance speedup as in this benchmark.

    ProxySQL overhead

    As mentioned above, ProxySQL adds additional functionality to the cluster. It can also add overhead, however. We would like to understand the expected performance penalty, so we compared the throughput and latency with and without ProxySQL.

    Out of box, the ProxySQL performance was not great and required additional tuning. 

    ProxySQL specific configuration:

    • Use connection through TCP-IP address, not through local socket
    • Adjust  mysql-max_stmts_per_connection variable for optimal value (default:50) – optimal – 1000
    • Ensure that “monitor@<host>” user has permissions as it’s important for proper handling of prepared statement.
      • CREATE USER ‘monitor’@‘172.30.%.%’ IDENTIFIED BY ‘monitor’;

    Throughput:

    Percona XtraDB Cluster on AWS 9

    Response time:

    Percona XtraDB Cluster on AWS 10

    ProxySQL performance penalty in throughput

    ProxySQL performance penalty IO1 i3
    large 0.97 0.98
    xlarge 1.03 0.97
    2xlarge 0.95 0.95
    4xlarge 0.96 0.93

     

    It appears that ProxySQL adds 3-7% overhead. I wouldn’t consider this a significant penalty for additional functionality.

    Summary

    Amazon instances

    First, the results show that instances based on General Purpose volumes do not provide acceptable performance and should be avoided in general for serious production usage. The choice is between IO provisioned instances and NVMe based instances.

    IO provisioned instances are more expensive, but offer much better performance than General Purpose volumes. If we also look at price/performance metric, IO provisioned volumes are comparable with General Purpose volumes. You should use IO provisioned volumes if you are looking for the functionality provided by EBS volumes.

    If you do not need EBS volumes, however, then i3 instances with NVMe volumes are a better choice. Both are cheaper and provide better performance than IO provisioned instances. Percona XtraDB Cluster provides data duplication on its own, which mitigates the need for EBS volumes to some extent.

    ProxySQL overhead

    We recommend using Percona XtraDB Cluster in combination with ProxySQL, as ProxySQL provides additional management and routing functionality. In general, the overhead for ProxySQL is not significant. But in our experience, however, ProxySQL has to be properly tuned — otherwise the performance penalty could be a bottleneck.

    Percona XtraDB Cluster scalability

    AWS has great capability to increase the instance size (both CPU and memory) if we exceed the capacity of the current instance. From our experiments, we see that Percona XtraDB Cluster can scale along with and benefit from increased instance size.

    Below is a chart showing the speedup in relation to the instance size:

    Percona XtraDB Cluster on AWS 11

    So increasing the instance size is a feasible strategy for improving Percona XtraDB Cluster performance in an AWS environment.

    Thanks for reading this benchmark! Put any questions or thoughts in the comments below.

    by Vadim Tkachenko at November 28, 2017 10:52 PM

    MariaDB AB

    MariaDB Server 10.2.11 now available

    MariaDB Server 10.2.11 now available dbart Tue, 11/28/2017 - 16:33

    The MariaDB project is pleased to announce the immediate availability of MariaDB Server 10.2.11. See the release notes and changelog for details and visit mariadb.com/downloads to download.

    Download MariaDB Server 10.2.11

    Release Notes Changelog What is MariaDB 10.2?

    The MariaDB project is pleased to announce the immediate availability of MariaDB Server 10.2.11. See the release notes and changelog for details.

    Login or Register to post comments

    by dbart at November 28, 2017 09:33 PM

    Peter Zaitsev

    Percona Monitoring and Management 1.5.0 Is Now Available

    Percona Monitoring and Management

    Percona announces the release of Percona Monitoring and Management 1.5.0 on November 28, 2017.

    This release focuses on the following features:

    • Enhanced support for MySQL on Amazon RDS and Amazon Aurora – Dedicated Amazon Aurora dashboard offers maximum visibility into key database characteristics, eliminating the need for additional monitoring nodes.  We renamed Amazon RDS OS Metrics to Amazon RDS / Aurora MySQL Metrics
    • Simpler configuration – Percona Monitoring and Management now offers easier configuration of key Amazon RDS and Amazon Aurora settings via a web interface
    • One-click data collection – One button retrieves vital information on server performance to assist with troubleshooting
    • Improved interface – A simple, consistent user interface makes it faster and more fluid to switch between Query Analytics and Metrics Monitor

    Highlights from our new Amazon RDS / Aurora MySQL Metrics dashboard:

    Shared elements for Amazon Aurora MySQL and RDS MySQL

    Amazon Aurora MySQL unique elements

    Amazon RDS for MySQL unique elements

    We’ve integrated Query Analytics into Metrics Monitor, and it appears as a separate dashboard known as PMM Query Analytics.

    With this release, Percona Monitoring and Management introduces a new deployment option via AWS Marketplace. This is in addition to our distribution method of Amazon Machine Images (AMI).

    We have upgraded Grafana and Prometheus in this release. PMM now includes Grafana 4.6.1. One of the most prominent features that the upgraded Grafana offers is the support of annotations. You can mark a point or select a region in a graph and give it a meaningful description. For more information, see the release highlights.

    Prometheus version 1.8.2, shipped with this release, offers a number of bug fixes. For more information, see the Prometheus change log.

    New features

    • PMM-434PMM enables monitoring of Amazon RDS and Amazon Aurora metrics
    • PMM-1133Query Analytics is available from Grafana as a dashboard
    • PMM-1470: Integrated Cloudwatch metrics into Prometheus
    • PMM-699: Combined AWS RDS and Amazon Aurora metrics into one dashboard
    • PMM-722: Distributed the MariaDB dashboard graph elements among other existing dashboards and removed the MariaDB dashboard. Further, we renamed the MyISAM dashboard  to MyISAM/Aria Metrics
    • PMM-1258: The DISABLE_UPDATES option enables preventing manual updates when PMM Server is run from a Docker container.
    • PMM-1500: Added InnoDB Buffer Disk Reads to graph InnoDB Buffer Pool Requests to better understand missed InnoDB BP cache hits

    Improvements

    • PMM-1577: Updated Prometheus to version 1.8.2
    • PMM-1603: Updated Grafana to version 4.6.1
    • PMM-1669: The representation of numeric values in the Context Switches graph in the System Overview dashboard was changed to improve readability.
    • PMM-1575: Templating rules were improved for the MyRocks and TokuDB dashboards so that only those instances with these storage engines are displayed

    Bug fixes

    • PMM-1082: The CPU Usage graph on the Trends dashboard showed incorrect spikes
    • PMM-1549: The authentication of the mongodb:queries monitoring service did not work properly when the name of the database to authenticate was not provided.
    • PMM-1673: Fixed display issue with Microsoft Internet Explorer 11

    by Borys Belinsky at November 28, 2017 12:56 PM

    November 27, 2017

    Peter Zaitsev

    autoxtrabackup v1.5.0: A Tool for Automatic Backups

    autoxtrabackup

    autoxtrabackupThere is a new version of the autoxtrabackup tool. In this post, I’ll provide some of the highlights available this time around.

    autoxtrabackup is a tool created by PerconLabs. We’ve now put out the 1.5.0 version, and you can test it further.

    Note: PerconaLabs and Percona-QA are open source GitHub repositories for unofficial scripts and tools created by Percona staff. While not covered by Percona support or services agreements, these handy utilities can help you save time and effort.

    autoxtrabackup is written in Python3 and hosted in PerconaLab (forked from Shako’s repo). Basically, this tool automates backup/prepare/copy-back actions. I want to talk about recent changes and additions.

    First of all, autoxtrabackup now has a --test_mode option, intended to test XtraBackup automation process.

    Here is the brief flow for this:

    • Clone percona-qa repo
    • Clone Percona Server for MySQL 5.6 and 5.7 from github.
    • Build PS servers in debug mode.
    • Get 2.3 and 2.4 versions of XtraBackup
    • Generate autoxtrabackup .conf files for each version of PS and XtraBackup
    • Pass different combination of options to PS start command and initialize PS servers each time with different options
    • Run sysbench against each started PS server
    • Take backup in cycles for each started PS + prepare
    • If make_slaves is defined, then create slave1 server from this backup (i.e., copy-back to another directory and start the slave from it)
    • Then take a backup, prepare and copy-back from this new slave1 to create slave2
    • Run pt-table-checksum on the master to check backup consistency

    I have prepared my environment, and now want to start --test_mode. Basically, it creates option combinations and passes them to the start script:

    2017-11-15 22:28:21 DEBUG    Starting cycle1
    2017-11-15 22:28:21 DEBUG    Will start MySQL with --innodb_buffer_pool_size=1G --innodb_log_file_size=1G
    --innodb_page_size=64K --early-plugin-load=keyring_file.so
    --keyring_file_data=/home/shahriyar.rzaev/XB_TEST/server_dir/PS131117-percona-server-5.7.19-17-linux-x86_64/mysql-keyring/keyring
    --log-bin=mysql-bin --log-slave-updates --server-id=1 --gtid-mode=ON --enforce-gtid-consistency --binlog-format=row

    So as you see, it is starting MySQL with --innodb_buffer_pool_size=1G --innodb_log_file_size=1G --innodb_page_size=64K. In cycle2, it will likely pick --innodb_buffer_pool_size=1G --innodb_log_file_size=1G --innodb_page_size=32K, and so on. It depends what you have passed in config:

    # Do not touch; this is for --test_mode, which is testing for XtraBackup itself.
    [TestConf]
    ps_branches=5.6 5.7
    gitcmd=--recursive --depth=1 https://github.com/percona/percona-server.git
    testpath=/home/shahriyar.rzaev/XB_TEST/server_dir
    incremental_count=3
    #make_slaves=1
    xb_configs=xb_2_4_ps_5_6.conf xb_2_4_ps_5_7.conf xb_2_3_ps_5_6.conf
    default_mysql_options=--log-bin=mysql-bin,--log-slave-updates,--server-id={},--gtid-mode=ON,--enforce-gtid-consistency,--binlog-format=row
    mysql_options=--innodb_buffer_pool_size=1G 2G 3G,--innodb_log_file_size=1G 2G 3G,--innodb_page_size=4K 8K 16K 32K 64K

    You can pass more options by changing the mysql_options in the config file. Also you can specify how many incremental backups you want by setting the incremental_count option. You can enable creating slaves from backup to test it as well, by enabling the make_slaves option. This is not recommended for daily usage. You can read more about it here: –test_mode.

    For daily backup actions, I have added the --tag and --show_tags options, which can be quite useful. They help you to tag your backups. Take a full backup:

    $ sudo autoxtrabackup --tag="My Full backup" -v
    -lf /home/shahriyar.rzaev/autoxtrabackup_2_4_5_7.log
    -l DEBUG --defaults_file=/home/shahriyar.rzaev/XB_TEST/server_dir/xb_2_4_ps_5_7.conf --backup

    Take an incremental one:

    $ autoxtrabackup --tag="First incremental backup" -v
    -lf /home/shahriyar.rzaev/autoxtrabackup_2_4_5_7.log
    -l DEBUG --defaults_file=/home/shahriyar.rzaev/XB_TEST/server_dir/xb_2_4_ps_5_7.conf --backup

    Take a second incremental one:

    $ autoxtrabackup --tag="Second incremental backup" -v
    -lf /home/shahriyar.rzaev/autoxtrabackup_2_4_5_7.log
    -l DEBUG --defaults_file=/home/shahriyar.rzaev/XB_TEST/server_dir/xb_2_4_ps_5_7.conf --backup

    Now you can use the --show_tags to list tags:

    $ sudo autoxtrabackup --show_tags
    --defaults_file=/home/shahriyar.rzaev/XB_TEST/server_dir/xb_2_4_ps_5_7.conf
    Backup              Type    Status  TAG
    -------------------------------------------
    2017-11-16_20-10-53 Full        OK  'My Full backup'
    2017-11-16_20-12-23 Inc         OK  'First incremental backup'
    2017-11-16_20-13-39 Inc         OK  'Second incremental backup'

    It would be quite nice if we could prepare those backups with a tag name. In other words, if I have a full backup and five incremental backups, what if I want to prepare until the second or third incremental, or just a full backup?

    Pass the tag name with the --prepare option, and it will do the trick:

    $ autoxtrabackup --tag="First incremental backup" -v
    -lf /home/shahriyar.rzaev/autoxtrabackup_2_4_5_7.log
    -l DEBUG --defaults_file=/home/shahriyar.rzaev/XB_TEST/server_dir/xb_2_4_ps_5_7.conf --prepare

    It will prepare the full and “First incremental backup” – the remaining incremental backups will be ignored.

    autoxtrabackup 1.5.0 also has a --dry_run option, which is going to show but not run exact commands. It is described here: –dry_run.

    How about autoxtrabackup 1.5.0 installation? You can install it from the source or use pip3:

    pip3 install mysql-autoxtrabackup

    For more please read: Installation.

    Do you want to enable encryption and compression for backups? Yes? You can enable this from the autoxtrabackup config as described here: Config file structure.

    You can enable taking partial backups again by editing the config: partial backups.

    autoxtrabackup 1.5.0 allows you to perform a partial recovery – i.e., restoring only a single specified table from a full backup. If the table was dropped,  autoxtrabackup will try to extract the create table statement from the .frm file using the mysqlfrm tool and then discard/import the tablespace from full backup. This is related to the transportable tablespace concept. You can read more here: restoring-single-table-after-drop.

    For a full list of available options, read the DOC: autoxtrabackup DOC.

    Thanks for reading! If you are going to try autoxtrabackup 1.5.0, don’t hesitate to provide some feedback!

    by Shahriyar Rzayev at November 27, 2017 06:39 PM