09:15 <cmouse> meh

09:15 <cmouse> tried to upgrade from 2.8.4 => 2.10.1

09:15 <cmouse> and got `builders must be a list or None` with no further explanation on checkConfig

09:15 <cmouse> lucky I have a test setup

18:03 <aakashjain> I tried to run 'buildbot upgrade-master' to upgrade from buildbot 1.8.2 to 2.10.1 (on a testing instance), with a sqllite database of ~33gb, it took ~7 hours. Is that normal?

18:03 <aakashjain> Would postgres database be more performant than this?

18:03 <aakashjain> I need to do similar thing for production instance which is ~500gb. I am wondering what I can do to reduce the down-time and make it faster.

18:05 <aakashjain> p12tic, tardyp ^

18:07 <p12tic> one option would be to upgrade incrementally

18:07 <p12tic> but first, question about the data: is it all needed? is it possible to delete build logs older than e.g. X months?

18:08 <aakashjain> p12tic: i might be able to delete some build logs, would that help in time taken for 'buildbot upgrade-master'?

18:09 <aakashjain> p12tic: what do you mean by incrementally?

18:11 <p12tic> e.g. upgrade to 2.x to do part of the upgrade, then to 2.x+1, etc.

18:11 <p12tic> i'm looking, in this case this won't help

18:12 <p12tic> the most likely reason for long upgrade is an addition of a index to steps table in v2.4

18:15 <p12tic> myself I didn't compare how much time postgres would take to add an index vs. sqlite. so my suggestion would be to take a backup and try

18:19 <aakashjain> p12tic: thanks. can you please point me to the v2.4 change which added indexes?

18:20 <p12tic> https://github.com/buildbot/buildbot/commit/16800f1a4

18:21 <aakashjain> p12tic: thanks. Also, would deleting old logs (using JanitorConfigurator) would help in reducing this upgrade time?

18:21 <p12tic> it seems that no, you would also need to delete steps

18:22 <p12tic> also it could be that sqlite just is plainly slow for indices

18:25 <aakashjain> p12tic: yeah. I wish JanitorConfigurator would be extended to delete old builders, builds and steps as well, e.g.: https://github.com/buildbot/buildbot/issues/4327

18:26 <p12tic> if that works for you, you could just do this manually by started_at column in steps. you'd still see builds and their results, but no information about steps

18:30 <aakashjain> p12tic: do what manually?

18:30 <p12tic> delete from DB

18:43 <shurdeek_> if I use a docker latent worker, can image and / or dockerfile be a renderable? So that the type of build can influence which environment is used.

18:50 <cmouse> yes

18:51 <shurdeek_> ok that makes things simpler

18:52 <shurdeek_> how about libvirt worker's xml argument, can that also be a renderable?

18:53 <cmouse> possibly not

18:53 <shurdeek_> and how about the locks arguments?

18:53 <cmouse> i can check

18:53 <shurdeek_> that would be great

18:54 <cmouse> i think libvirt has no renderable xml

18:54 <cmouse> which is bit sad

18:54 <cmouse> this is the same thing as was with openstack worker, i had to add bunch of renderable stuffs to it to make it actually usable.

18:56 <cmouse> shurdeek_: https://github.com/buildbot/buildbot/commit/bebb5f93db4fa36a37f685ea558db31a28e5c77e

18:56 <cmouse> gives some hints how to do this.

18:57 <shurdeek_> yes the docker worker also has render props

18:57 <shurdeek_> I think I'll add that on my todo list once my other libvirt work is merged

18:59 <shurdeek_> thank you it would have taken me longer to find the answers on my own

19:10 <p12tic> shurdeek_: I think that PR depends only on you testing on your setup with multiple libvirt connections.

19:50 <aakashjain> p12tic: I got this error on buildbot upgrade-master command. is it safe to ignore?

19:51 <aakashjain> /usr/local/lib/python3.6/site-packages/buildbot/db/migrate/versions/052_cascading_set_null.py:81: UserWarning: Inconsistent data found in DB: table 'logs', deleting invalid rows

19:51 <aakashjain> 'Inconsistent data found in DB: table %r, deleting invalid rows' % t)

19:51 <aakashjain> upgrade complete

19:51 <cmouse> aakashjain: i would not worry too much.

19:51 <cmouse> aakashjain: it's logs table

19:54 <p12tic> the message says that there were rows in logs table with NULL stepid columns. These are harmless, as one can't see such rows from within the web UI anyway

19:54 <p12tic> the error message should have been beter

19:54 <p12tic> s/beter/better/

20:11 <aakashjain> p12tic: ok, thanks. postgres was much more performant (~15 minutes for ~500 gb data). Also maybe sqllite was slow because because when I run 'buildbot upgrad-master', my ssh session got disconnected, and when i re-run the command it gave various errors about database being locked.