MySQL Proxy and master.info (contribution 2 of N)

This time I'll write about a nice featured we now have on the MySQL Proxy.

Parsing master.info
The master.info is a file that the MySQL server creates when you are running a slave server, you can read the manual for more details.
And now you can use the MySQL Proxy to read that file and parse the data from it.

Background
It all started when Jan emailed the proxy discuss mailing with the Changelog from 0.6.1 to 0.7.0. The part that got my attention was:
"...
Core
[skip]
* added parser for master.info files"
... "
As I was working on a Lua script that did some simulation, I asked Jan how that worked, and I found out that it was only implemented on C-land, and only compatible with MySQL server 5.1. That meant you could not write a Lua script and use the parser we had.



Nothing like motivation
So I went ahead and asked how much work it would take to implement this feature and I was pointed to the lib/mysql-proto.c file. As I really wanted to have this implemented, I couldn't resist and made a local branch to work on it.

I first looked at that whole file, trying to figure out how things worked and after reading lib/mysql-proto.c I found out I also needed to look at src/network-mysqld-masterinfo.c. After some more reading, I was ready to write some code.

Thought process
Well, I was ready to modify what was there. The way I usually add code when I'm not very familiar with it is by modifying the current code, little by little, compile, test, fix what I broke and continue. I do this a few times until I have something new working :).

This time I took this function:

static int lua_proto_get_ok_packet (lua_State *L) {
 size_t packet_len;
 const char *packet_str = luaL_checklstring(L, 1, &packet_len);
 network_mysqld_ok_packet_t *ok_packet;
 network_packet packet;
 GString s;
 int err = 0;

 s.str = (char *)packet_str;
 s.len = packet_len;

 packet.data = &s;
 packet.offset = 0;

 ok_packet = network_mysqld_ok_packet_new();

 err = err || network_mysqld_proto_get_ok_packet(&packet, ok_packet);
 if (err) {
  network_mysqld_ok_packet_free(ok_packet);

  luaL_error(L, "%s: network_mysqld_proto_get_ok_packet() failed", G_STRLOC);
  return 0;
 }

 lua_newtable(L);
 LUA_EXPORT_INT(ok_packet, server_status);
 LUA_EXPORT_INT(ok_packet, insert_id);
 LUA_EXPORT_INT(ok_packet, warnings);
 LUA_EXPORT_INT(ok_packet, affected_rows);

 network_mysqld_ok_packet_free(ok_packet);

 return 1;
}

I duplicated it and renamed it "lua_proto_get_masterinfo_string". I chose that function as it had some "LUA_EXPORT_INT" lines which sounded like what I needed.

As you can see form here I started by commenting out some lines and renaming some function calls and other lines of code.

I also added a line to "luaL_reg mysql_protolib" which is where things really get exposed to Lua (I found this out by searching the whole branch for "from_ok_packet" which I found on test/unit/lua/mysql-proto.lua)

I then compiled the new code and wrote a simple Lua script to test this (which was not the best way, but more on that later).
At this point all I wanted to test was that I could call "get_masterinfo()" from Lua, and it worked!

I then removed the lines that I comentd out and after showing Jan where I was, he suggested renaming the function, so I did.

Where I should've started.
We all now that we should first have a test, and then code, well, I wasn't sure what I was doing so I started the other way, but eventualy I got to writing the test for this piece of code.
There was already a lua unit test for the mysql-proto.c file, so I added some tests for the master.info parsing.

At that point, it only worked for master.info files from MySQL 5.1, so the next step was to add support for 4.1 and 5.0.

The way I implemnted the suport for 4.1 and 5.0 master.info files took me a few days, I really wanted to return a NULL (nil in Lua) value for "master_ssl_verify_server_cert" if it was not present, but I really did not know how to do it. Either Jan or Kay came to the rescue there and we ended up with this.

Basically, if you don't export it, it will be NULL. And having the unit test made it so much easier to test the changes.

And as I was done with this, I went ahead and proposed a merge and a few days later it was part of the Main Proxy code.

Next?
So far we can parse a master.info file, but we cannot "create" one, so this is next, I want to have a function that would create the master.info file, I know I could just use the io library from Lua, but it will be more fun to add a "to_masterinfo_string()" function on Lua land :)

Thanks and enjoy!