Chris CalenderMariaDB 5.5.39 Overview and Highlights (6.8.2014, 00:40 UTC)

MariaDB 5.5.39 was recently released (it is the latest MariaDB 5.5), and is available for download here:

https://downloads.mariadb.org/mariadb/5.5.39/

This is a maintenance release, and so there are not too many big changes of note, just a number of normal bug fixes. However, there are a few items worth mentioning:

If interested, the official MariaDB 5.5.39 release notes are here:

https://mariadb.com/kb/en/mariadb/development/release-notes/mariadb-5539-release-notes/

And the full list of fixed bugs and changes in MariaDB 5.5.39 can be found here:

https://mariadb.com/kb/en/mariadb/development/changelogs/mariadb-5539-changelog/

Hope this helps.

 

Link
Monty SaysLogs and more logs, who has time to read them ? (5.8.2014, 19:07 UTC)
While working on some new features in MariaDB 10.1, I noticed that a normal user couldn't disable the slow query log, which I thought was a bit silly.

While implementing and documenting this feature, I noticed that the information about the different logs is quite spread around and it's not that trivial to find out how to enable/disable the different logs.

To solve this, I created a new MariaDB kb entry, "Overview of the MariaDB logs that I hope MariaDB and MySQL users will be find useful.

Here follows a copy of the kb entry. If you have any comments or things that could be added, please do it in the kb entry so that it will benefit as many as possible!

Overview of MariaDB logs

There are many variables in MariaDB that you can use to define what to log and when to log.

This article will give you an overview of the different logs and how to enable/disable logging to these.

The error log

  • Always enabled
  • Usually a file in the data directory, but some distributions may move this to other locations.
  • All critical errors are logged here.
  • One can get warnings to be logged by setting log_warnings.
  • With the mysqld_safe --syslog option one can duplicate the messages to the system's syslog.

General query log

  • Enabled with --general-log
  • Logs all queries to a file or table.
  • Useful for debugging or auditing queries.
  • The super user can disable logging to it for a connection by setting SQL_LOG_OFF to 1.

Slow Query log

The binary log

  • Enabled by starting mysqld with --log-bin
  • Normally only used on machines that are, or may become, replication masters.
  • Binary log files are mainly used by replication and can also be used with --binlog-ignore-db=database_name or --binlog-do-db=database_name.
  • The super user can disable logging for a connection by setting SQL_LOG_BIN to 0. However while this is 0, no changes done in this connection will be replicated to the slaves!
  • For examples, see

Truncated by Planet PHP, read more at the original (another 784 bytes)

Link
Chris CalenderMySQL 5.6.20 Overview and Highlights (5.8.2014, 16:53 UTC)

MySQL 5.6.20 was recently released (it is the latest MySQL 5.6, is GA), and is available for download here.

For this release, there is 1 “Security Fix”, 1 “InnoDB Important Change”, and 7 “Functionality Added or Changed” fixes, all of which should be read in case they might affect you (though for this release, these mostly appear to be minor – some [default] changes, build notes/changes, and deprecations):

  1. Security Fix: The linked OpenSSL library for the MySQL 5.6 Commercial Server has been updated from version 1.0.1g to version 1.0.1h. Versions of OpenSSL prior to and including 1.0.1g are reported to be vulnerable to CVE-2014-0224. This change does not affect the Oracle-produced MySQL Community build of MySQL Server 5.6, which uses the yaSSL library instead.
  2. InnoDB Important Change: Redo log writes for large, externally stored BLOB fields could overwrite the most recent checkpoint. The 5.6.20 patch limits the size of redo log BLOB writes to 10% of the redo log file size. The 5.7.5 patch addresses the bug without imposing a limitation. For MySQL 5.5, the bug remains a known limitation. As a result of the redo log BLOB write limit introduced for MySQL 5.6, innodb_log_file_size should be set to a value greater than 10 times the largest BLOB data size found in the rows of your tables plus the length of other variable length fields (VARCHAR, VARBINARY, and TEXT type fields). Failing to do so could result in “Row size too large” errors. No action is required if your innodb_log_file_size setting is already sufficiently large or your tables contain no BLOB data. (Bug #69477)
  3. Replication: The new system variable binlog_impossible_mode controls what happens if the server cannot write to the binary log, for example, due to a file error. For backward compatibility, the default for binlog_impossible_mode is IGNORE_ERROR, meaning the server logs the error, halts logging, and continues updates to the database. Setting this variable to ABORT_SERVER makes the server halt logging and shut down if it cannot write to the binary log. (Bug #51014)
  4. CMake support was updated to handle CMake version 3.
  5. New Debian7, Ubuntu12.04, and Ubuntu14.04 distribution support that was introduced with 5.6.17 now comes with the platform-specific packaging source placed under the packaging directory, in the deb-precise, deb-wheezy, and deb-trusty directories.
  6. Support for LinuxThreads has been removed from the source code. LinuxThreads was superseded by NPTL in Linux 2.6. (Bug #72888)
  7. By default, mysql_install_db creates a my.cnf file in the installation base directory using a template. This may be undesireable for some deployments. To enable this behavior to be suppressed, mysql_install_db now supports a –keep-my-cnf option to preserve any existing my.cnf file and not create a new my.cnf file. (Bug #71600)
  8. The mysqlhotcopy utility is now deprecated and will be removed in a future version of MySQL. Among the reasons for this: It works only for the MyISAM and ARCHIVE storage engines; it works on Unix but not Windows. Alternatives include mysqldump and MySQL Enterprise Backup.
  9. The timed_mutexes system variable has no effect and is deprecated.

In addition to those, there were 52 other bug fixes:

  • 11 InnoDB
  • 09 Replication
  • 01 Partitioning
  • 31 Miscellaneous

There were 2 regression bugs fixed, so check if they might have affected you:

  1. InnoDB: A regression introduced in MySQL 5.6.5 would cause full-text search index tables to be created in the system tablespace (space 0) even though innodb_file_per_table was enabled.
  2. When a SELECT included a derived table in a join in its FROM list and the SELECT list included COUNT(DISTINCT), the COUNT() returned 1 even if the underlying result set was empty.

Conclusions:

I’ve not too much to say about this release regarding whether one should upgrade or not. I will say that if you are running a *Commercial* version of MySQL 5.6, pre-5.6.20, and you use SSL, then you should upgrade. Aside from that, it more depends if you’re affected by one of these bugs or not.

The full 5.6.20 changelogs can be viewed here (which has more details about all of the bugs listed above):

http://dev.mysql.com/doc/relnotes/mysql/5.6/en/news-5-6-20.html

Hope this helps. :)

 

Link
Peter ZaitsevNew in Percona Replication Manager: Slave resync, Async stop (5.8.2014, 14:39 UTC)

Percona Replication Manager (PRM) continues to evolve and improve, I want to introduce two new features: Slave resync and Async stop.

Slave resync

This behavior is for regular non-gtid replication.  When a master crashes and is no longer available, how do we make sure all the slaves are in a consistent state. It is easy to find the most up to date slave and promote it to be the new master based on the master execution position, the PRM agent already does that but how do we apply the missing transactions to the other slaves.

In order to solve that problem, I modified a tool originally written by Yelp, that outputs the MD5 sums of the payload (XID boundaries) and the commit positions of a binlog file. It produces an output like:

root@yves-desktop:/home/yves/src/PRM/percona-pacemaker-agents/tools/prm_binlog_parser# ./prm_binlog_parser.static.x86_64 /opt/prm/binlog.000382 | tail -n 5
53844618,E5A35A971445FC8E77D730932869F2
53846198,E37AA6704BE572727B342ABE6EFA935
53847779,B17816AC37424BB87B3CD55226D5EB17
53848966,A7CFF846D281DB363FE4192C38DD7743
53850351,A9C5B7FC24A1BA3B97C332CC362633CE

Where the first field is the commit position and the second field, the md5 sum of the payload. The only type of transaction that is not supported is “LOAD DATA INTO”. Of course, since it relies on XID values, this only works with InnoDB. It also requires the “log_slave_updates” to be enabled. The sources and some static binaries can be found in the tools directory on the PRM github, link is at the bottom.

So, if the agent detects a master crash and the prm_binlog_parser tool is available (the prm_binlog_parser_path primitive parameter), upon promotion, the new master will look at its binary logs and publish to the pacemaker CIB, the last 3000 transactions. The 3000 transactions limit is from bash, the command line must be below 64k. With some work this limit could be raised but I believe, maybe wrongly, that it covers most of the cases. The published data in the CIB attribute also contains the corresponding binlog file.

The next step happens on the slaves, when they get the post-promotion notification. They look for the MD5 sum of their last transaction in the relay log, again using the prm_binlog_parser tool, find the matching file and position in the last 3000 transactions the new master published to the CIB and reconfigure replication using the corrected file and position.

The result is a much more resilient solution that helps having the slave in a consistent state. My next goal regarding this feature will be to look at the crashed master and attempt to salvage any transactions from the old master binlog in the case MySQL crashed but Pacemaker is still up and running.

Async_stop

The async_stop feature, enabled by the “async_stop” primitive parameter, allows for faster failover. Without this feature, when stopping mysqld, PRM will wait until is confirmed stopped before completing a failover. When there’re many InnoDB dirty pages, we all know that stopping mysql can take many minutes. Jervin Real, a colleague at Percona, suggested that we should fake to Pacemaker that MySQL is stopping in order to proceed faster with failover. After adding some safe guards, it proved to be a very good idea. I’ll spare you of the implementation details but now, if the setting is enabled, as soon as mysqld is effectively stopping, the failover completes. If it happens that Pacemaker wants to start a stopping MySQL instance, a very usual situation, an error will be generated.

PRM agents, tools and documentation can be found here: https://github.com/percona/percona-pacemaker-agents

The post New in Percona Replication Manager: Slave resync, Async stop appeared first on MySQL Performance Blog.

Link
Colin CharlesPercona Live London Call for Presentations (5.8.2014, 08:01 UTC)

Europe traditionally doesn’t have many MySQL-dedicated conferences, which is why I personally enjoy Percona Live London, now in its 2014 Edition. This year it happens November 3-4, and the call for presentations is still open — till August 17th.

DSCF1396The topic list is growing as the MySQL ecosystem matures: DevOps, cloud, security, case studies and what’s new are things you don’t often see. Tutorials are also welcome, of course.

Location-wise, London can’t be beat. And happening at Gloucester Road, you’re on the District/Circle/Picadilly lines to go to many fun places.

If you don’t want to present, do attend – registration is open. Early-bird (ending August 31st) conference & tutorials will set you back £425.00 and if you just want to attend the conference only, its £235.00 (VAT and fees excluded). A steal if you ask me!

See you there!

Link
Peter ZaitsevQ&A: Putting MySQL Fabric to use (4.8.2014, 18:56 UTC)

Percona MySQL webinar followup: Q&AMartin Arrieta and I gave an online presentation last week on “Putting MySQL Fabric To Use.” If you missed it, you can find a recording and the slides here, and the vagrant environment we used plus a transcript of the commands we ran here (be sure to check out the ‘sharding’ branch, as that’s what we used during the webinar).

Thank you all for attending and asking interesting questions. We were unable to answer all of them in the scheduled time, so here are our replies to all the questions.

What is GTID? And how does it relate to MySQL Fabric?
GTID stands for Global Transaction Identifier, and MySQL Fabric requires MySQL replication to be configured to used this in order to work. This means that it won’t work with MySQL versions previous to 5.6. You can find more information in the manual, and in the upcoming Percona webinar, “Using MySQL Global Transaction IDs in Production,” that our colleague Stephane Combaudon will present on August 27.

During any issue, does MySQL Fabric internally do a switch over or failover?
MySQL Fabric will only detect failures and initiate failovers for a group that has been activated. Otherwise, any changes must be made manually through the mysqlfabric command.

For an alter table, how do you avoid replication lag?
We think using pt-online-schema for the alter table is a good way to avoid or minimize this problem.

Are there benchmarks available for MySQL Fabric?
We’re not aware of any, but once we wrap up our current series of blog posts on the topic, we plan to do some benchmarks ourselves. Our interest lies in the overhead the connectors can place vs a standard connector accessing a single MySQL Server, but if you have other ideas, let us know in the comments and we’ll consider adding them to our list.

Can MySQL Fabric be used to handle Big Data?
We think the HA features are usable regardless of data set size, with the caveat that MySQL Fabric does not (currently) handle provisioning the data of the managed instances. It just configures and points replication as required, but you’re responsible for making sure all servers have the same set of data.

We feel that the sharding features, however, may not be a good fit for working with very large data sets yet, specially because of the use of mysqldump/mysql on the shard split/move operations. However, since sharding usually goes hand in hand with big data (a data set size too large is one of the reasons to shard after all) we’re sure this will get improved in the future. Or, someone with some python skills can adjust the scripts to use a more efficient mechanism to move data from one server to another, like Percona XtraBackup.

Does sharding require many changes to application coding (database code on shards etc) at the MySQL Fabric level? Should we sit with the developers/architects to help them understand tables, their usage, and to plan sharding?
Sharding with MySQL Fabric is not transparent (and we’d say sharding, in general, is typically not transparent) so some application coding and DBAs/Devs interaction will be needed. But you should be working close to your developers and architects (or DBAs and sysadmins, depending on which side of the counter you stand) anyway :)

Would you say that MySQL Fabric is a good HA replacement removing the need for VIPs?
We don’t see them as mutually exclusive. Simplifying, the HA features of MySQL Fabric are just another way to manage replication between a set of MySQL servers. It doesn’t force any way to make writable and readable nodes available to clients. With some coding, you can use MySQL Fabric together with VIPs, or dynamic DNS entries, or a load

Truncated by Planet PHP, read more at the original (another 7785 bytes)

Link
Peter ZaitsevAdvanced MySQL Query Tuning (Aug. 6) and MySQL 5.6 Performance Schema (Aug. 13) webinars (4.8.2014, 13:56 UTC)

I will be presenting two webinars in August:

This Wednesday’s webinar on advanced MySQL query tuning will be focused on tuning the “usual suspects”: queries with “Group By”, “Order By” and subqueries; those query types are usually perform bad in MySQL and add an additional load as MySQL may need to create a temporary table(s) or perform a filesort. New this year: I will talk more about new MySQL 5.6 features that can increase MySQL query performance for subqueries and “order by” queries.Percona MySQL webinars

Next week’s performance schema webinar will be focused on practical use of Performance Schema in MySQL 5.6. I will show real-world examples of MySQL monitoring and troubleshooting using MySQL 5.6 Performance Schema. Performance Schema can give you a lots of valuable information about MySQL internal operations. For example, if using a multi-tenant shared MySQL installation, Performance Schema can give you a lots of extensive information about your MySQL load: breakdown by user, queries, tables and schema (see more examples in my previous blog post: Using MySQL 5.6 Performance Schema in multi-tenant environments).

 

 

The post Advanced MySQL Query Tuning (Aug. 6) and MySQL 5.6 Performance Schema (Aug. 13) webinars appeared first on MySQL Performance Blog.

Link
Federico RazzoliMariaDB and Stored Procedures: errata & repository (3.8.2014, 09:20 UTC)
A
Link
Chris CalenderTroubleshooting TokuDB Corruption (2.8.2014, 03:42 UTC)

I recently ran across some TokuDB table corruption, which was not easily identifiable at first, and the error log entry was not too verbose either, so I wanted to share that experience here.

Basically, TokuDB crashed, and then mysqld had problems restarting afterward. Just for reference, the error log had the following in the stack trace:

/usr/lib64/mysql/plugin/ha_tokudb.so(+0x71c48)[0x7fb25be75c48]
/usr/lib64/mysql/plugin/ha_tokudb.so(+0x71cbd)[0x7fb25be75cbd]
/usr/lib64/mysql/plugin/ha_tokudb.so(_Z29toku_deserialize_bp_from_disk...
/usr/lib64/mysql/plugin/ha_tokudb.so(_Z23toku_ftnode_pf_callback...
/usr/lib64/mysql/plugin/ha_tokudb.so(_Z30toku_cachetable_pf_pinned_pair..
/usr/lib64/mysql/plugin/ha_tokudb.so(_Z24toku_ft_flush_some_child...
/usr/lib64/mysql/plugin/ha_tokudb.so(_Z28toku_ftnode_cleaner_callback...
/usr/lib64/mysql/plugin/ha_tokudb.so(_ZN7cleaner11run_cleaner...
/usr/lib64/mysql/plugin/ha_tokudb.so(+0xcfeee)[0x7fb25bed3eee]
/lib64/libpthread.so.0(+0x79d1)[0x7fb39310b9d1]
/lib64/libc.so.6(clone+0x6d)[0x7fb391828b5d]

The crash was not really the problem here; that is another story.

But rather mysqld not starting back up was an issue.

Given this crash, and mysqld not starting back up, we needed to check every fractal tree file in the data directory.

I’m not an expert with this, but what I did, and you may find helpful too, is to iteratively check each fractal tree file with “tokuftdump –nodata tokutablename”.

If it does not return anything, then there is a problem with the table – not too scientific ;) – but was effective in this case.

If you have just a couple files to check, this is easy enough to run per file. If there are a lot to check, then a simple shell loop with the check (and output the name at the time) can help.

Any corrupt keys should be then dropped and recreated.

That should get you back going.

On a side note, when looking more into “tokuftdump –nodata”, I did run across these 2 commands (one-liners), which allow you to check the compression format of the tables also using tokuftdump –nodata. Thus, it illustrates the use of this command for another feature, and it’s easily modifiable if you’re interested:

Pre-MariaDB 5.5.37:

mysql -sNe 'SELECT dictionary_name, internal_file_name FROM
information_schema.tokudb_file_map WHERE dictionary = "main"
OR dictionary LIKE "key-%"'
|perl -F'\t' -ane '@out=qx(tokuftdump --nodata --rootnode /var/lib/mysql/$F[1]);
foreach $ftdump (@out) { if($ftdump=~/^ compression_method=([0-9]+)$/)
{ print $F[0]."\t".$1."\n"; last } }'

MariaDB 5.5.37+ (b/c “dictionary” column changed to “table_dictionary_name”):

mysql -sNe 'SELECT dictionary_name, internal_file_name FROM
information_schema.tokudb_file_map WHERE table_dictionary_name = "main"
OR table_dictionary_name LIKE "key-%"'
|perl -F'\t' -ane '@out=qx(tokuftdump --nodata --rootnode /var/lib/mysql/$F[1]);
foreach $ftdump (@out) { if($ftdump=~/^ compression_method=([0-9]+)$/)
{ print $F[0]."\t".$1."\n"; last } }'

For reference, these are documented here.

Hope this helps.

 

Link
Chris CalenderTroubleshooting TokuDB ERROR 1126 – API Version Mismatch or bitmap_free (2.8.2014, 03:40 UTC)

When setting up TokuDB, you may encounter error 1126.

I’ve seen 2 recent invocations of it, so I wanted to share them both here in case you run into this issue:

MariaDB [(none)]> install soname 'ha_tokudb';
ERROR 1126 (HY000): Can't open shared library
'/usr/lib/mysql/plugin/ha_tokudb.so' (errno: 2, undefined symbol: bitmap_free)
MariaDB [(none)]> install soname 'ha_tokudb';
ERROR 1126 (HY000): Can't open shared library 'ha_tokudb.so'
(errno: 8, API version for STORAGE ENGINE plugin TokuDB not
supported by this version of the server)

The latter is a bit more descriptive, but the former is fairly cryptic.

Given the latter, as you may have guessed it, if you run into either of these, you have the wrong version of ha_tokudb.so in your plugin directory – that is, it is not the correct version for your MariaDB/MySQL.

In both cases, 10.0.11 was used. In the first case, the ha_tokudb.so was from 5.5.37. In the second case, the ha_tokudb.so was from 10.0.12.

Hope this helps.

 

Link
LinksRSS 0.92   RDF 1.
Atom Feed