Tag Archives: mysql

installing and tuning mariadb on osx

Wanted to install MariaDB on OSX. Mostly because I was running one in an Ubuntu VM and it seemed slow. Running a some dozens inserts taking several seconds.

First install homebrew from http://brew.sh.
The process is to invoke some cryptic looking command line sentence, which downloads a ruby script over the internet and runs that with the ruby interpreter. Had a look at the script and decided just to install as usual (woohoo for security).

After this: “brew install mariadb”. Wish it was so simple. Once it is installed run “mysql.server start” to start it. Seemed to work, huh. But how do I modify the configuration? The internetz seem to point to /etc/my.sql file that defines the config. No such file. Then after some googling, I get the info I should copy a file called my-default.cnf to /etc/my.cnf.

Well, the dir mentioned where this default file should be does not exist. Instead it seems homebrew installs all software to /usr/local/Cellar directory. So MariaDB is installed in /usr/local/Cellar/mariadb/VERSION. And the config files to copy are under this directory in “support-files”. But there is no my-default.cnf. Only “my-small.cnf”, “my-medium.cnf”, “my-large.cnf”, “my-huge.cnf”. And looking into these it seems that they are a bit out of date as the “huge” talks about a server with 2GB of RAM.

So copy one to /etc/my.cnf. But it seems to have no effect.. More googling. There is a command “my_print_defaults –help” that prints in the end (few paragraphs up) the directories where the DB looks for the configurations. This points to “/usr/local/etc/my.cnf”. This file actually exists but is practically empty. Whee. Well, because it is empty the DB ends up using default configurations. Overwriting something seems to actually work. So we can define settings in this file and they get used. Whee. For options we can look at the my-small.cnf etc. files.

Finally, we can restart the server with “mysql.server restart”. Which ended up failing after all my different configuration attemps at one point. “brew uninstall mariadb” did not work either, as after re-install it still failed to restart. The only thing it prints is the informative message “ERROR”. Whooppee. Google to the rescue and the result is the log is in “/usr/local/var/mysql/your_computer_name.local.err”, where the real cause can be found.

Anyway, had to remove the dir /usr/local/var/mysql and then restart it all (or was it reinstall?). Now it works.. Whooppeee.

And when it finally works? It turned out to be as slow as the Ubuntu version so what was the point.. Finally tried to optimize with setting InnoDB transactions to write once a second (not after all commits) and to write all commits in a single transaction. Helped maybe 20-30%. Still way slow, so whatever..?

python3 mysql connector and pip install procedures

Trying to learn some Python and use some of the nice and shiny libraries available for it.

Problem: Need to use MySQL to store some data. Have to figure out how to connect and use the MySQL DB from Python3.

Several places suggest using a Python package called MySQLdb. Seems fine but does not support Python3 at this time. Installation seems a bit of a pain as well with downloading this and that from different places on different platforms. Also, if there is an “official” version/package I would prefer that. Turns out is an “official” package called MySQL Connector from Oracle. Trying to use that.

Problem again: The “official” site (http://dev.mysql.com/downloads/connector/python/ at this time) hosts some “installer” files to download. So I download one and run the installer. Nothing seems to happen as my Python3 install still does not recognize the package. Why not just “pip3 install xyz”? Such lovely simple thing for me..

So, it appears there is something that can actually be installed with pip. On the internetz I finally find some references to a command as “pip install mysql-connector-python –allow-external mysql-connector-python”. But what does this mean? Well I change pip to pip3 to get Python3 and the “pip3 install mysql-connector-python” seems obvious. The install command with a package name matching the official package. But what is “–allow-external” and why is mysql-connector-python repeated twice?

Some of this info I found in the docs, others on stackoverflow, some is my guessing. So here goes, hoping for corrections where relevant.

The “pip” or “pip3” commands download and install stuff from the Python package index a.k.a. “pypi” (located at https://pypi.python.org/pypi/). The “install” command for “pip” then looks for the given package name in the “pypi” repository and installs one if found. However, only some of the actual install files are located in the “pypi” repository. Some files are just references to an external site. So the flag “–allow-external” tells the “pip” command to also install a package even if it is not in “pypi”. In this case, “pypi” only contains the reference to the package. For security purposes this should still be fine as the “pypi” entry still should contain the checksum of the external file so no malicious stuff should be downloaded with any higher probability than getting one from “pypi”.

And how does it work to find something on “pypi”? There is a search functionality for packages at the “pypi” repository pages. If we type “mysql” in there, we get a hit in the top 10 which has a name “mysql-connector-python 2.0.3”. Looks interesting for this case. If we click on the link, we get an info page. This has information such as author etc that can help (Oracle in this case) to figure out if it is what we are looking for. At the bottom of the page there is a “DOAP record” link. This gives us an XML file which in this case shows the name as “mysql-connector-python”. So this is where I guess the match is made for the “install” command. Then there is a version number for upgrades. Finally, there is a download link with an MD5 checksum embedded in the end. So guessing again, this is where the “–allow-external” goes to download the installed external files and the MD5 checksum is what it uses to check the file is valid.

Whooppee, did we learn something useful? Hope so..