python – balazspocze.me

Continuous feedback from jenkins console output

Posted on 2015-12-18 by banyek

This one really was a pain in the ass for a long time.

Jenkins is great, not only for building projects, but I start the most of the ops related tasks here. All the projects have their own workspace, I can browse the previous outputs, see if a task failed or not, and I can easily share the results at the end of the task.

I had only one problem with that: the long-running tasks weren’t shown on console output until they finished. I tried to solve that, but it was only annoying, not a showstopper, so I let that go, because it didn’t look like an easy win, and I didn’t want to spend too much time on that.

But last week I created a job for online altering the database and – surprise! – pt-online-schema-change was continuously updated the console log screen, so I started to google out what is happening.

Continue reading “Continuous feedback from jenkins console output”

My first impressions about Go language

Posted on 2015-10-13 by banyek

I am fascinated. Maybe that should be enough, but I guess I have to write a bit more here because we are not on twitter.

I spent a few days to get know Go language, and now I am more than satisfied. I mean, all the project ideas which are floating in my head should be written in Go.

First of all, I have rewritten Mambo-collector to go (https://github.com/banyek/mambo) because I have faced some serious errors when I used it on CentOS 7 – I blame systemd -: If the process was running as root then after a few days of data collecting, killing that process was lead to restart the entire system, which is not a bug, it is a catastrophe. I tried to debug it several ways, but I am not sure where the problem is, it could be at the ‘loghandler’ redirection or any other place in python-daemon, or it is simply there is a buffer inside which overflows – I don’t really care, because mambo was just a proof of concept – what I used in production. I decided to abandon the project, and rewrite it in a language, what is designed for what mambo is: a system process with a high amount of parallel data processing. In the original version, I spawned an os thread for every query which will run (and we know that there is no real multithreading in python because of the global interpreter lock, at the end, building a multithreaded application in python is just an illusion. It is good for helping you to imagine what will happen inside your the computer, but you cannot expect real multithreading. It wasn’t a real problem because I only ran/run a few queries with mambo – replication lag, all processes inside mysql, active processes inside mysql – but if there would be a need to increase the number of queries that could lead to trouble. And it’s needless to say if you have a lot of time.sleep in your program, that is not the wisest use of your CPU resources: if a CPU is sleeping, then you waste resources.I have enough resources to let waste a bit, but it always disturbed me.

At the end, I decided to rewrite mambo in go.

Continue reading “My first impressions about Go language”

MySQL variables module for ansible

Posted on 2013-08-13 by banyek

I’ve created and another module for this besides the mysql_replication named mysql_variables.

The purpose of this module to get/set variables from running MySQL servers, and avoid to use shell module for this, because that is a bit frustrating, and there are long commands to run.

For example, when you wanna to set read_only mode via shell, you can do that like this:

Continue reading “MySQL variables module for ansible”