This one really was a pain in the ass for a long time.
Jenkins is great, not only for building projects, but I start the most of the ops related tasks here. All the projects have their own workspace, I can browse the previous outputs, see if a task failed or not, and I can easily share the results at the end of the task.
I had only one problem with that: the long-running tasks weren’t shown on console output until they finished. I tried to solve that, but it was only annoying, not a showstopper, so I let that go, because it didn’t look like an easy win, and I didn’t want to spend too much time on that.
But last week I created a job for online altering the database and – surprise! – pt-online-schema-change was continuously updated the console log screen, so I started to google out what is happening.
At the end, the keyword which needed to be google is the output buffering. Jenkins console will show the results continuously, if the output buffer is flushed, or the buffering is skipped.
If you want to run Perl, you can use
With python, you can invoke the interpreter with
#/usr/bin/env python -u
Or you can use the PYTHONUNBUFFERED environment variable.
With ruby scripts you can use:
$stdout.sync = true
If you use bash, you can use stdbuf utility for changing stdout/stderr buffering.