Aug 282021
 

I like readability. I like it in variable names. I prefer something descriptive and easy to ready.

In in this post:

  • FreeBSD 13.0-RELEASE-p4
  • poudriere 3.3.7

I recently started getting messages like this from my daily poudiere build:

[00:00:00] Warning: Using '-' in a SETNAME is not recommended as it causes ambiguities with parsing the build name of 122amd64-default-master-list
[00:00:00] Warning: Using '-' in a SETNAME is not recommended as it causes ambiguities with parsing the build name of 122i386-default-master-list-i386
[00:00:00] Warning: Using '-' in a SETNAME is not recommended as it causes ambiguities with parsing the build name of 13amd64-default-master-list
[00:00:00] Warning: Using '-' in a SETNAME is not recommended as it causes ambiguities with parsing the build name of 13i386-default-master-list-i386

I could just rename the sets, everything would be rebuilt, but then clients would be stuck pointing at the old no-longer built trees. To fix this easily, I will use symlinks. This is what clients use now:

[dan@pkg01:~]: $ pkg -vv | grep url
    url             : "pkg+http://fedex.int.unixathome.org/packages/13amd64-default-master-list/",

I don’t want to change that on 80-90 clients. I might update them over time, but not now.

As I mentioned on Twitter, I’m going to us symlinks for that.

Renaming the build lists

Here are my build lists, only two offenders:

[dan@pkg01:/usr/local/etc/poudriere.d/buildlists]: $ ls -l
total 49
-rw-r--r--  1 root  wheel  9176 Aug 26 14:41 master-list
-rw-r--r--  1 root  wheel  3215 Jun 11 18:55 master-list-i386
-rw-r--r--  1 root  wheel   263 Mar 21 14:25 mysql57
-rw-r--r--  1 root  wheel   263 Mar 21 14:25 mysql80
-rw-r--r--  1 root  wheel   269 Jun 11 10:14 pg10
-rw-r--r--  1 root  wheel   269 Jun 11 10:14 pg11
-rw-r--r--  1 root  wheel   269 Jun 11 10:14 pg12
-rw-r--r--  1 root  wheel   402 Jun 11 10:14 pg13
-rw-r--r--  1 root  wheel   269 Jun 11 10:14 pg96
-rw-r--r--  1 root  wheel   272 Jun 24  2020 unifi

I’m going to do this rename:

[dan@pkg01:/usr/local/etc/poudriere.d/buildlists]: $ sudo svn mv master-list primary
A         primary
D         master-list
[dan@pkg01:/usr/local/etc/poudriere.d/buildlists]: $ sudo svn mv master-list-i386 primaryi386
A         primaryi386
D         master-list-i386
[dan@pkg01:/usr/local/etc/poudriere.d/buildlists]: $ 
[dan@pkg01:/usr/local/etc/poudriere.d/buildlists]: $ svn st
D       master-list
        > moved to primary
D       master-list-i386
        > moved to primaryi386
A  +    primary
        > moved from master-list
A  +    primaryi386
        > moved from master-list-i386
[dan@pkg01:/usr/local/etc/poudriere.d/buildlists]: $ sudo svn ci -m 'rename, without using -, to keep poudriere quiet'
Deleting       master-list
Deleting       master-list-i386
Adding         primary
Adding         primaryi386
Transmitting file data .done
Committing transaction...
Committed revision 12265.

I keep my build lists in a repo.

Updating the jobs

Fortunately, the changes there are minor:

[dan@pkg01:/root/bin]: $ grep master-list *
config.sh:I386_SETS="master-list-i386"
config.sh:NEW_SIMPLE_SETS="master-list mysql57 mysql80 pg96 pg10 pg11 pg12 pg13 unifi"
[dan@pkg01:/root/bin]: $ 

I have only two changes to make, as you can see above.

The results are shown here:

[dan@pkg01:/root/bin]: $ grep primary *
config.sh:I386_SETS="primaryi386"
config.sh:NEW_SIMPLE_SETS="primary mysql57 mysql80 pg96 pg10 pg11 pg12 pg13 unifi"

But what jobs run?

These are the jobs which run, well, it’s one script, but many builds:

$ sudo crontab -l -u root
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/home/dan/bin
# use /bin/sh to run commands, overriding the default set by cron
SHELL=/bin/sh
# mail any output to `dan', no matter whose crontab this is
MAILTO=dan

18  4 * * * /usr/bin/lockf -t 0 /tmp/.poudriere.build /root/bin/poudriere-builds.sh
[dan@pkg01:/root/bin]: $ 

What’s this lockf crap you might ask? Read more here. Use it for anything which should only ever have one instance running.

What’s in that script?

$
$ cat poudriere-builds.sh
#!/bin/sh

/root/bin/poudriere-refresh-trees           2>&1 >> /usr/local/poudriere/data/cronjob-logs/poudriere-refresh-trees.log
/root/bin/poudriere-builds-122-amd64.sh     2>&1 >> /usr/local/poudriere/data/cronjob-logs/poudriere-builds-122-amd64.log
/root/bin/poudriere-builds-122-i386.sh      2>&1 >> /usr/local/poudriere/data/cronjob-logs/poudriere-builds-122-i386.log
/root/bin/poudriere-builds-13-amd64.sh      2>&1 >> /usr/local/poudriere/data/cronjob-logs/poudriere-builds-13-amd64.log
/root/bin/poudriere-builds-13-i386.sh       2>&1 >> /usr/local/poudriere/data/cronjob-logs/poudriere-builds-13-i386.log

First, I refresh all my poudriere ports trees:

$ cat poudriere-refresh-trees
#!/bin/sh

. /root/bin/config.sh

# updates the ports trees
for tree in ${TREES}
do
  ${POUDRIERE} ports -u -p ${tree}
done

Wait, that’s in the configuration file?

$ cat /root/bin/config.sh
#!/bin/sh

POUDRIERE="/usr/local/bin/poudriere"

TREES="default testing"

I386_SETS="primaryi386"

NEW_SIMPLE_SETS="primary mysql57 mysql80 pg96 pg10 pg11 pg12 pg13 unifi"

Here is one of the actual build scripts:

[dan@pkg01:/root/bin]: $ cat /root/bin/poudriere-builds-122-amd64.sh
#!/bin/sh

. /root/bin/config.sh

for set in ${NEW_SIMPLE_SETS}
do
  ${POUDRIERE} bulk -j 122amd64 -p default -z ${set} -f /usr/local/etc/poudriere.d/buildlists/${set}
done

I build all of those sets for 122amd64. The other scripts are all similar to that one.

The i386 scripts looks like this:

$ cat poudriere-builds-122-i386.sh
#!/bin/sh

. /root/bin/config.sh

for set in ${I386_SETS}
do
  ${POUDRIERE} bulk -j 122i386 -p default -z ${set} -f /usr/local/etc/poudriere.d/buildlists/${set}
done

What’s next?

Tomorrow I will create symlinks to the new repos. Or I might do that via an nginx alias. I’m not sure yet. I want to keep the old stuff around, but have the clients use the new stuff, transparently. An alias would do that. A symlink requires that the existing stuff is renamed or goes away. Keeping the original name means I can use poudriere to clean up later.

See also the follow-up post: poudriere: when renaming sets, also rename *-make.conf files etc in which I tell Nginx where to find the new stuff so the clients do not need to be modified.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive