Subscribe to RSS Subscribe to Comments

My So-Called Blog

No Silver Bullet

Giuseppe Maxia’s new article at the MySQL DevZone rightly points out that using partitions doesn’t work by magic; as with any other enhancement, it requires a bit of know-how to realise its value. MySQL 5.1 Partitions in Practice reviews some of the basics and points out a few gotchas (if you want to partition on a DATE or DATETIME column, pay close attention here). He then provides some realistic examples showing how you actually get partition pruning working for your queries and backs it up with some interesting performance figures.

On a related note, I recently reorganised the MySQL Manual’s discussion of functions in partitioning expressions to get rid of the multiple listings (at Giuseppe’s behest). Now there’s a single Supported Functions list, which we hope you’ll find less confusing and more useful.

Partitioning Optimizations

We’ve been getting lots of questions about how MySQL 5.1 optimises queries on partitioned tables. Recently, we added some new information about this to our Internals Manual provided by one of our developers, Sergey Petrunia. (I hope that my edits and formatting haven’t messed up anything — if there are any problems with what’s been published in the Internals Manual, then the fault’s probably mine and not his.)

The new section can be accessed here, and includes coverage of the following topics:

  • An overview of how MySQL performs partition pruning
  • What partitioning and subpartitioning intervals are, and why they’re important
  • Converting WHERE and ON clauses into intervals
  • Partition selection
  • Where to find partition pruning and selection code in the 5.1 sources

Sergey’s included several helpful examples and diagrams as well. Even if you’re not interested in hacking the source, you might find this material useful for understanding better how MySQL Partitioning works “behind the scenes”, so be sure to check it out.

The Truth About Keys and Partitioning

I recently discovered that I’d made a huge blunder in the Partitioning Limitations section of the MySQL 5.1 Manual. I’d had the idea that unique keys don’t effect your choice of partitioning columns unless there’s no primary key.

My latest revision of the documentation for this limitation isn’t much better.

Then, as I was sitting here just now trying to persuade my daughter that she really ought to have something for lunch besides marshmallows, it came to me.

It’s very simple, really.

For any user-partitioned table in MySQL 5.1:

  1. If a table has any unique keys, then all columns used in the partitioning expression must also be part of any and all of these keys.
  2. By definition, a primary key is a unique key.
  3. If the table has no unique keys, then you don’t have to worry about this limitation.
  4. By definition, a primary key is a unique key.

At the moment, I’ve a very important date for tea, but as soon as I get the chance, I’ll update the Manual with this much simpler version of something I’ve tried to make very complicated for a long time.

Yet Another Partitioning Article…

The article that Peter Gulutzan and I wrote a couple of weeks ago is now online:

MySQL 5.1 New Features: MySQL Partitions

Here’s a brief outline:

  • A Grammar Description That’s Expressed Differently From The Manual
    • Partitioning Types
    • Creating Partitioned Tables
    • Partitioning Keys, Primary Keys, and Unique Keys
    • Altering Partitioning Tables
    • Performance Considerations
  • Advantages of Partitioning
    • Partition Pruning
    • Maintenance of Big Tables
  • Doing It With Dates And Times
  • Some Simple Speed Tests
  • Comparison With Other Database Systems
  • Examples:
    • Creating a Table With 500 Partitions
    • Patching Maintenance Mistakes
    • Searching One Partition At A Time
  • References

Since this is a rather long article (ca. 30 pages), I’ve decided to make it even more worthwhile reading by introducing an exciting Guess The Author Contest, which I’m sure will provide countless hours of enjoyment for the entire family: The first person to guess correctly which of us (Jon Stephens or Peter Gulutzan) wrote which sections of the article will receive a modest assortment of quality Australian-made Freddo chocolates from me. Send guesses to me in an email or post them in a comment to this article. Deadline: Midnight, Sunday 27 August 2006 GMT +10.00. Employees/Contractors of MySQL AB/Inc/GmbH/KK and their families are not eligible to participate in this contest. Neither Peter Gulutzan nor MySQL AB/Inc/GmbH/KK have any involvement whatsoever in this contest, so don’t bother asking them about it, please. Winner is responsible for taxes and customs duties. Not responsible for mail that is late, stolen, misdirected, or allowed to stay out in the sun for too long. Prohibited where void. Cash value of prize offering is approximately AUD10.00. I’ll post the name and URL (if desired) of the winner on this site on or before Monday 28 August 2006. Or - if there’s no winner by then - I’ll eat the chocolates myself, and tell you how nice they were.

Need Some Space?

Someone at work pointed out to me this blog post about what to do when disk partitions housing MySQL databases fill up, with the caption, “An idea for [the MySQL 5.1 Partitioning] Docs?”

My reply was, “Nope”.

Well, I was wrong. In a way. This guy’s talking about disk partititions, not partitioned tables. And using symlinks to point to data or database directories is not particularly new or exciting. However, partitioning in MySQL 5.1 provides a kind of cool new way to move table data and index files pretty easily.

The DATA DIRECTORY and INDEX DIRECTORY options have been around for CREATE TABLE for quite some time, but they’re not supported for ALTER TABLE - if you try, this is what happens:

 
mysql> ALTER TABLE t1
    ->     DATA DIRECTORY = '/home/jon/data'
    ->     INDEX DIRECTORY = '/home/jon/data';
Query OK, 0 rows affected, 2 warnings (0.02 sec)
Records: 0  Duplicates: 0  Warnings: 0
 
mysql> SHOW WARNINGS;
+---------+------+------------------------------–+
| Level   | Code | Message                        |
+---------+------+------------------------------–+
| Warning |    0 | DATA DIRECTORY option ignored  |
| Warning |    0 | INDEX DIRECTORY option ignored |
+---------+------+------------------------------–+
2 rows in set (0.00 sec)
 

Well, that’s not very happy, is it? However, in MySQL 5.1, you can do this:

 
ALTER TABLE t1
    PARTITION BY HASH(col) (
        PARTITION p0
            DATA DIRECTORY = '/path/to/dir'
            INDEX DIRECTORY = '/path/to/dir'
);
 

In this case, MySQL dutifully creates copies of the table’s .MYD and .MYI files in the new location, then replaces the originals with symlinks. Hey, presto! You’ve just moved all the bulky bits of the table. And if the path points to a location on a different disk, then you’ve just freed up a chunk of space on the original drive.

And yes, it is perfectly legal to create a partitioned table with only one partition.

If you don’t want the table to remained partitioned, here’s all you have to do:

ALTER TABLE t1
    REMOVE PARTITIONING;

Now the partitioning is gone, but the symlinks remain. And the data and index files remain where you’ve moved them by means of the ALTER TABLE ... PARTITION BY. Too easy!

The fine print: The table schema doesn’t matter, except that if the table has a primary key (or a unique key in the event there’s no primary key), then any column used for partitioning that table must be part of that key.

MySQL 5.0.45-communityPHP 5.2.3