#archlinux32 | Logs for 2023-06-11

Back
[06:46:30] -!- abaumann has joined #archlinux32
[06:46:30] <buildmaster> Hi abaumann!
[06:46:30] <buildmaster> !rq abaumann
[06:46:31] <phrik> buildmaster: <abaumann> *abaumann does some hackeroo on his rsync-woodo-script..
[08:00:11] <abaumann> mmh. all package builds fail the mirrored_source_by_hash and the haskell_without_check (the latter is obsolete anyway). Then they build with build support only..
[08:07:55] <abaumann> + build_dir='/var/lib/archbuild/extra-staging-'\''pentium4'\''/slave1'
[08:08:10] <abaumann> instead of /var/lib/archbuild/slave1/extra-staging-'\''pentium4'\''
[08:08:20] <abaumann> there seems to be minor inconsistenties here
[08:09:08] <abaumann> build_dir="/var/lib/archbuild/${build_command%-build}/$(whoami)"
[08:09:37] <abaumann> strange, ls /var/lib/archbuild/slave1
[08:09:38] <abaumann> core-staging-with-build-support-i686 extra-staging-with-build-support-pentium4
[08:09:40] <abaumann> core-staging-with-build-support-pentium4
[08:09:59] <abaumann> they should really be of the form slaveN/xxx
[08:45:33] <abaumann> ah.
[08:45:34] <abaumann> archbuild_chroots='/var/lib/archbuild/slave1'
[08:45:43] <abaumann> and the whomai is the last level.
[08:45:54] <abaumann> ok, back to the miracle, why everything builds in build_support mode..
[08:56:33] <abaumann> oups. Fixed the bug..
[08:58:58] <abaumann> mmh. almost all packages I build have an upstream git sha of 0000000000000000000000000000000000000000..
[10:09:00] -!- abaumann has quit [Remote host closed the connection]
[10:46:05] <girls> are you sure, that it's the _upstream_ hash, not the mod hash?
[10:47:07] <bill-auger> i dont know if this is related, but i have seen that before
[10:47:39] <girls> the upstream hash should only be all zeros, if it's a pure arch32 package
[10:48:04] <girls> on the other hand, the our hash should be all zeros for all packages, where we don't have any modifications
[10:48:08] <bill-auger> i see that in git hooks - a hash of all zeros, on the left, indicates a new branch - a hash of all zeros, on the right, indicates a deleted branch
[10:48:23] <girls> ah, yes
[10:48:35] <girls> but that's not, what goes into these hashes in our db
[10:48:49] <girls> we use the content of the files in upstream's state repository
[11:30:18] <girls> 32
[11:30:25] <girls> 5555555555555
[11:39:30] -!- drathir_tor has quit [Remote host closed the connection]
[11:40:07] -!- drathir_tor has joined #archlinux32
[11:48:20] -!- drathir_tor has quit [Remote host closed the connection]
[11:48:49] -!- drathir_tor has joined #archlinux32
[11:50:23] <girls> ... children ...
[11:50:35] <girls> I should lock my keyboard more often
[12:28:58] <KitsuWhooa> So, that 10GB needed?
[12:29:03] <KitsuWhooa> 2023-06-11 15:22:38: building package "xonotic-data" (revisions ab3932838c63c7299a20e6cf01b540db552a1dc3 0000000000000000000000000000000000000000, repository extra, straw :mirrored_source_by_hash:) for pentium4 ...builder/bin/build-packages: line 615: echo: write error: No space left on device
[12:29:06] <KitsuWhooa> :p
[12:32:57] -!- GNUtoo has joined #archlinux32
[12:32:58] <girls> :(
[12:33:19] <girls> I was just looking at what my build slave currently needs
[12:33:26] <girls> apparently some packages need more - sry
[14:09:58] -!- socksinspace has quit [Quit: WeeChat 3.0]
[14:10:16] -!- socksinspace has joined #archlinux32
[15:24:08] <KitsuWhooa> Do we really have 28k pending packages to be built? :p
[16:13:02] <girls> yes, unfortunately
[16:14:19] <KitsuWhooa> I wonder if I should spin up more builders given that the CPU is still mostly idling
[16:14:36] <girls> you can do that, we only need the ssh keys
[16:14:48] <KitsuWhooa> there was a mention of using chroots
[16:14:50] <girls> but cpu is not the only thing - keep an eye on ram, too
[16:14:53] <KitsuWhooa> but didn't go into details
[16:14:59] <girls> yes, but it's one task per ssh key
[16:15:17] <KitsuWhooa> Yeah, and each one has its own ssh key, right?|
[16:15:21] <KitsuWhooa> I just don't understand how I'd set it up
[16:15:24] <girls> exactly
[16:15:33] <KitsuWhooa> the guide doesn't say anything beyond "fill in paths"
[16:16:27] <KitsuWhooa> VM has 24 cores and 32GB ram. I could probably split that to 3 builders with makeflags -j8
[16:16:38] <girls> yes, that sounds sufficient
[16:17:12] <KitsuWhooa> Right now I'm just manually running the builder from bash. Would I run it in a different way by specifying chroots?
[16:17:30] <girls> doesn't matter
[16:17:39] <girls> I'm running one from a systemd service and one from bash
[16:17:49] <girls> I believe, abaumann also runs his from a systemd service
[16:17:54] <KitsuWhooa> yes, but how do I run them specifying a different builder
[16:18:41] <KitsuWhooa> unless I'm misunderstanding how this is done
[16:18:55] <KitsuWhooa> in https://bbs.archlinux32.org I see "Also, configure separate archbuild chroots for each of them:"
[16:18:56] <phrik> Title: How to create a build slave? / Building / Arch Linux 32 Forum (at bbs.archlinux32.org)
[16:18:58] <KitsuWhooa> which I don't really know what it means
[16:20:22] <girls> hmm, I have never set it up that way
[16:20:28] <girls> abaumann is the expert, here
[16:21:08] <girls> the only way, I see, to make this work out-of-the-box is to clone the repository multiple times and run the script from different users
[16:21:23] <girls> although, on second thought, the cloning might be unnecessary
[16:23:17] <girls> ah, no, there's *one* lock file, due to which you need to clone the repo
[16:23:31] <girls> let me see, if I can change that
[16:24:47] <KitsuWhooa> I just don't understand what I'm supposed to do, that's all :p
[16:25:49] <girls> so you need multiple system users, because upstream's build commands will lock on the system user (i.e. one user can only run one build per repository)
[16:26:09] <girls> and then it's merely a matter of starting multiple build-packages scripts, one from each user
[16:26:30] <girls> but as it currently looks, one also needs to start them in different builder repositories, because otherwise all will try to lock the same file
[16:26:35] <girls> but I can fix the latter
[16:26:45] <KitsuWhooa> I see
[16:32:37] <girls> ok, 60fd162d7b33a33a9c4c1a22a3b2a1f9298327e1 should have all, that you need
[16:34:11] <girls> ahh, you need to configure different ssh keys
[16:34:23] <girls> this will also not work, when you run within the same repository
[16:34:38] <KitsuWhooa> I can run them as separate users with separate clones. It should be fine
[16:34:45] <girls> ok :)
[16:37:37] <girls> I could add an option to overwrite the config dir by an environment variable, though - this should simplify stuff a lot
[16:38:37] <KitsuWhooa> I wonder if I can just copy the .gnupg folder
[16:38:50] <girls> symlink should also work
[16:39:44] <girls> ah, no - you run as different users
[16:39:51] <girls> you need to copy and change ownership
[16:39:59] <girls> ... or you do a bind mount >:-)
[16:40:43] <KitsuWhooa> I'll just copy it :p
[16:40:48] <girls> :)
[16:48:43] <girls> abaumann, KitsuWhooa: you can now specify a different configuration location by setting the CONFIG_DIR environment variable. If all works as intended, one can now run multiple build slaves from within the same git "builder" repository.
[16:48:51] <KitsuWhooa> too late:p
[16:48:55] <KitsuWhooa> but appreciated
[16:48:56] <girls> :'-(
[16:49:06] <girls> all those electrons waster for NOTHING
[16:49:09] <girls> ;)
[16:49:29] <KitsuWhooa> heh
[16:49:48] <girls> hopefully, I remember next time someone sets up multiple builders ...
[17:05:23] <girls> abaumann: haskell_without_check is actually still useful. It will skip check() on haskell and python (!) packages. Since it's only invoked together with clean_chroot, it is also never any additional work.
[17:06:37] <girls> similar things go for mirrored_source_by_hash: it will try to download the source from the hashing mirror and simply skip the build, if it couldn't do so
[17:07:02] <girls> tl;dr: I won't remove those straws :)
[17:13:58] <KitsuWhooa> girls: remind me, what do I do with builder/bin/build-packages: line 574: extra-staging-with-build-support-pentium4-build: command not found
[17:20:26] <KitsuWhooa> a lot of ruby packages seem to be failing with that
[17:22:28] <KitsuWhooa> same for builder/bin/build-packages: line 574: extra-staging-with-build-support-i686-build: command not found
[17:22:39] <KitsuWhooa> I don't think I broke/changed anything
[17:28:51] <KitsuWhooa> I think I'm not building anything at the moment...
[17:29:53] <KitsuWhooa> maybe symlink extra-staging to /usr/bin/staging-with-build-support-i686-build ?
[17:34:28] <KitsuWhooa> I'll try installing devtools32 manually
[18:03:52] <girls> you need to create all those symlinks
[18:04:06] <girls> have a look at what symlinks devtools32 already has in /usr/bin
[18:04:32] <girls> devtools32 is currently incomplete in this regard
[18:04:42] <girls> it should be symlinks to archbuild IIRC
[18:30:47] <KitsuWhooa> some packages failed because sudo
[18:30:50] <KitsuWhooa> been slowly fixing that
[18:31:24] <KitsuWhooa> that said
[18:31:26] <KitsuWhooa> this is a new one
[18:31:30] <KitsuWhooa> ==> Installing packages to /var/lib/archbuild/extra-staging-with-build-support-pentium4/root
[18:31:32] <KitsuWhooa> error: no usable package repositories configured.
[18:35:39] <KitsuWhooa> girls: ^
[18:36:20] <KitsuWhooa> for example 21:35:21 <buildmaster> any/ruby-benchmark is broken (says build-tasossah-1): https://archlinux32.org
[18:36:21] <girls> ah, and you need to copy the pacman-*.conf
[18:36:55] <KitsuWhooa> in /usr/local/share/devtools ?
[18:37:09] <girls> yeah :)
[18:37:16] <KitsuWhooa> I only have one pacman conf in there
[18:37:26] <KitsuWhooa> ....
[18:37:27] <KitsuWhooa> I'm dumb
[18:37:29] <KitsuWhooa> nevermind that
[18:37:31] <KitsuWhooa> :p
[18:47:22] <KitsuWhooa> girls: extra-staging-with-build-support-pentium4.conf is 0 bytes
[18:47:39] <KitsuWhooa> what am I supposed to copy exactly?
[18:54:03] <KitsuWhooa> Yeah, running make in the devtools32 source tree results in 0 byte files being generated
[19:09:54] <girls> we should really fix devtools32 ...
[19:10:14] <girls> ah, it's /usr/share/devtools/
[19:10:56] <KitsuWhooa> I installed from source
[19:11:02] <girls> ah, ok
[19:11:08] <KitsuWhooa> and make generates 0 bytes conf files
[19:11:18] <girls> :-/
[19:11:23] <girls> then abaumann broke it :D
[19:12:03] <girls> I have the version from [releng] installed
[19:12:18] <girls> but this is still broken, one needs to copy/symlink the pacman.conf's
[19:12:38] <girls> and one needs to add core-testing, core-staging, extra-testing and extra-staging to the configs
[19:12:47] <girls> ... all not very user friendly, currently ...
[19:13:15] <KitsuWhooa> https://tasossah.com
[19:13:16] <phrik> Title: devtools32 config files (at tasossah.com)
[19:14:12] <girls> yeah, it creates the correct files, but they are empty :(
[19:14:33] <girls> ah, also the names are off, if I'm not mistaken
[19:14:51] <KitsuWhooa> just let me know what to do at this point. I removed the devtools32 package and make/make installed from source and I am getting the no usable package repositories configured
[19:14:52] <girls> nvm - I'm off: it's the new package structure
[19:16:51] <girls> install this package: https://mirror.archlinux32.org
[19:17:40] <KitsuWhooa> after manually cleaning up the local installation from source?
[19:17:45] <girls> yes
[19:17:59] <girls> and then you need a small sed one-liner
[19:18:12] <KitsuWhooa> I did run that one you sent me last time
[19:18:19] <girls> yes
[19:18:21] <KitsuWhooa> I assume re-run it
[19:18:21] <girls> ok
[19:18:30] <KitsuWhooa> and then make symlinks to archbuild for the ones misisng?
[19:18:32] <girls> and you need to copy some pacman-*.confs
[19:18:38] <girls> that too, yes
[19:18:46] <KitsuWhooa> okay, let me get to that point
[19:19:41] <girls> for s in {core,extra}-staging{,-with-build-support}-build do ln -s archbuild /usr/bin/$s; done
[19:21:20] -!- drathir_tor has quit [Ping timeout: 240 seconds]
[19:22:02] <girls> for s in {core,extra}-staging{,-with-build-support}{i486,i686,pentium4}; do cp /usr/share/devtools/pacman-${s#*-*-} /usr/share/devtools/pacman-$s; done
[19:22:35] <girls> ah, no, typo
[19:22:50] <KitsuWhooa> Okay, package installed, ran the pacman sed line
[19:22:57] <KitsuWhooa> 22:19:40 <girls> for s in {core,extra}-staging{,-with-build-support}-build do ln -s archbuild /usr/bin/$s; done <-- now this, correct?
[19:23:49] <girls> for s in {core,extra}-staging{,-with-build-support}-{i486,i686,pentium4}.conf; do cp /usr/share/devtools/pacman-staging-${s#*-*-} /usr/share/devtools/pacman-$s; done
[19:24:02] <girls> the ln should be correct, yes
[19:24:18] <KitsuWhooa> [root@Arch64 ~]# for s in {core,extra}-staging{,-with-build-support}-build do ln -s archbuild /usr/bin/$s; done
[19:24:20] <KitsuWhooa> -bash: syntax error near unexpected token `done'
[19:24:27] <KitsuWhooa> I assume missing ; before the do
[19:24:44] <KitsuWhooa> yup, that did it
[19:25:00] <KitsuWhooa> okay, ran the cp one too
[19:29:16] <girls> yeah, you're right
[19:29:29] <girls> btw: sry for the latency, having to care about a couple children in parallel
[19:29:35] <KitsuWhooa> don't worry about it
[19:29:38] <KitsuWhooa> I appreciate you taking the time :p
[19:29:45] <KitsuWhooa> is there anything else or should I fire the builders up?
[19:29:57] <girls> I think, it should work (tm)
[19:30:02] <KitsuWhooa> let's find out
[19:31:09] <KitsuWhooa> girls: builder/bin/build-packages: line 574: extra-staging-with-build-support-i686-build: command not found
[19:34:13] <KitsuWhooa> you forgot to include the architectures I guess
[19:35:08] <KitsuWhooa> I'll make those manually
[19:36:32] <girls> indeed, i486, i686 and pentium4 were missing in the links
[19:41:54] <KitsuWhooa> looks like we're getting somewhere now
[19:42:05] <KitsuWhooa> it's normal that the first two straws(?) fail, right?
[19:42:27] <KitsuWhooa> the third one succeeds and builds stuff
[19:42:29] <girls> normally, it should be able to build without any straws
[19:42:42] <girls> or are these only the source straws?
[19:42:58] <KitsuWhooa> it tries mirrored_source_by_hash first and it fails
[19:43:10] <girls> ah, ok, that's no problem
[19:43:18] <KitsuWhooa> then clean_chroot:haskell_without_check: fails
[19:43:24] <KitsuWhooa> and finally with_build_support succeeds
[19:43:24] <girls> it doesn't even try to build with this one - this merely tries to download the source by hash
[19:43:27] <KitsuWhooa> ah
[19:43:43] <girls> hmm, clean_chroot should not fail, actually
[19:43:58] <girls> the third one is with build-support repository
[19:44:08] <KitsuWhooa> I don't know what the difference is :p
[19:44:13] <girls> this means, we have some package in build-support, which is essential for building
[19:44:22] <KitsuWhooa> clean_chroot seems to work for i686
[19:44:25] <KitsuWhooa> but not for pentium4
[19:44:28] <girls> this is meant as a shim repository for build slaves only
[19:44:32] <KitsuWhooa> ah
[19:44:58] <girls> clean_chroot builds normally, but prunes the chroot before building
[19:45:34] <girls> `build-packages --help` gives a brief explanation of the straws
[19:45:43] <KitsuWhooa> I see
[19:45:48] <KitsuWhooa> is there any way to get logs for a specific one?
[19:45:55] <KitsuWhooa> to ese why the clean_chroot one fails on pentium4
[19:46:12] <girls> you mean from the buildmaster?
[19:46:16] <KitsuWhooa> *to see
[19:46:18] <KitsuWhooa> from anywhere
[19:46:18] <girls> no, it only saves the last log
[19:46:26] <girls> your slave has them all
[19:46:34] <KitsuWhooa> where? :p
[19:46:35] <girls> they're in work/tmp.build-packages.*
[19:46:43] <girls> but only for the current build
[19:46:46] <KitsuWhooa> yes
[19:46:53] <girls> the directory gets pruned after finishing the build
[19:46:58] <KitsuWhooa> if I tail all the files there, I just see the current build in action
[19:47:46] <KitsuWhooa> it looks like the log file gets deleted between the attempts?
[19:48:09] <girls> I'm not sure
[19:48:20] <girls> sounds reasonable, but I always thought, they would be kept
[19:48:49] <KitsuWhooa> well, I'll leave it as is for now
[19:49:10] <girls> you can specify a straw with -s
[19:49:33] <girls> then you will upload this specific error to the buildmaster :)
[19:49:41] <girls> very cumbersome :D
[19:50:27] <girls> ah, no: the build logs should be kept locally: they're time stamped
[19:50:37] <KitsuWhooa> builder/bin/build-packages -s clean_chroot:haskell_without_check ?
[19:50:43] <girls> if there is only one *.build-log, then maybe it didn't even get to the build step
[19:50:54] <girls> colon at the start and end is important
[19:50:57] <KitsuWhooa> yeah I'm only seeing one
[19:51:06] <girls> -s :clean_chroot:haskell_without_check:
[19:51:15] <KitsuWhooa> ah
[19:51:30] <girls> I was lazy, there's a regex matching on ':clean_chroot:', etc.
[19:53:42] <KitsuWhooa> builder/bin/build-packages: line 574: extra-staging-pentium4-build: command not found
[19:55:10] <KitsuWhooa> some more symlinks later...
[19:55:50] <girls> for s in {core,extra}-staging{,-with-build-support}-{i486,i686,pentium4}-build do ln -s archbuild /usr/bin/$s; done
[19:56:10] <girls> you should already have some of them
[19:56:38] <KitsuWhooa> missing ; but yes, re-ran that
[19:56:43] <KitsuWhooa> s/re-//
[19:56:48] <girls> damn copy paste error
[19:57:00] <KitsuWhooa> it decided to build i686 now, so I can't tell if it worked
[19:57:01] <KitsuWhooa> :p
[19:57:25] <girls> :D
[20:01:31] <KitsuWhooa> looks like it works
[20:01:38] <girls> \o/
[20:22:37] <girls> time for my bed - good night!
[20:22:48] <KitsuWhooa> night!
[20:22:53] <girls> (but I guess, it's not yet evening in your timezone, KitsuWhooa?)
[20:23:05] <KitsuWhooa> I'm an hour ahead
[20:23:07] <KitsuWhooa> :p
[20:23:25] <girls> ah, I see :)
[20:38:30] -!- drathir_tor has joined #archlinux32
[20:46:45] -!- drathir_tor has quit [Remote host closed the connection]
[20:52:09] -!- drathir_tor has joined #archlinux32
[21:47:07] -!- bill-auger has quit [Ping timeout: 240 seconds]
[23:09:59] <KitsuWhooa> Python/meson needs to be fixed. So many packages are failing because of it