This post is all about creating technical debt. If you accept that, go for it. I’m avoiding porting my FreshPorts scripts into SITEPERL. Why? I’ll migrate them to SITEPERL after BSDCan & PGCon. Right now, I need to get the servers upgraded from Perl 5.24 to Perl 5.26, because 5.24 is deprecated.
FreshPorts uses Perl for processing incoming commits and for various administrative backend tasks. Everything on the front end (website) is PHP and HTML. Perl is great at text processing, and text processing is at the heart of what drives FreshPorts.
Lately, my daily security run output FreeBSD emails have been mentioning:
perl5-5.24.4: Tag: expiration_date Value: 2019-05-09 perl5-5.24.4: Tag: deprecated Value: Support ends three years after .0 release, please upgrade to a more recent version of Perl
I’ve been using Perl 5.24 for a while. It is the default on my Poudriere builds.
I decided to upgrade the old FreshPorts server to Perl 5.26, which is now the default on FreeBSD, since 19 April 2018.
Immediately after upgrading, I started getting this problem:
Can't locate port.pm in @INC (you may need to install the port module) (@INC contains: /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26) at hourly_stats.pl line 10. BEGIN failed--compilation aborted at hourly_stats.pl line 10. Can't locate db_utils.pm in @INC (you may need to install the db_utils module) (@INC contains: /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26) at test-master-port.pl line 15. BEGIN failed--compilation aborted at test-master-port.pl line 15. Can't locate port.pm in @INC (you may need to install the port module) (@INC contains: /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26) at /usr/websites/freshports.org/scripts/daily_rendering_times.pl line 10. BEGIN failed--compilation aborted at /usr/websites/freshports.org/scripts/daily_rendering_times.pl line 10.
Those are from three separate scripts.
Options
I started investigating and found a few options:
- I could alter all the perl scripts with
use lib ‘path.to.files’
But that hardcodes the location in the script. Less than ideal. - I could set PERL5LIB in an environment variable.
- I could use -I when the scripts are invoked.
I decided, based on suggestions from mat_ on IRC to use the SITECUSTOMIZE option when building Perl.
Enabling SITECUSTOMIZE
I added this to the supernew-make.conf file for my poudriere build:
lang_perl5.26_SET+=SITECUSTOMIZE
Then I rebuilt Perl and installed it. Then I searched for perl SITECUSTOMIZE and found
the INSTALL docs for Perl 5.26. What I learned there: When enabled, this makes perl run $sitelibexp/sitecustomize.pl before anything else. This script can then be set up to add additional entries to @INC.
Great. So what do I have now? This is how you see @INC for your perl:
$ perl -E 'map{say}@INC'
/usr/local/lib/perl5/site_perl/mach/5.26
/usr/local/lib/perl5/site_perl
/usr/local/lib/perl5/5.26/mach
/usr/local/lib/perl5/5.26
As ChoHag pointed out: ‘say for @INC’ is a simpler idiom than ‘map{say}@INC’. Which means this also works:
$ perl -E 'say for @INC' /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26 /usr/websites/freshports.org/scripts
You can also use this command to see various environment variables related to Perl installed on your system:
$ env -i perl -V
Summary of my perl5 (revision 5 version 26 subversion 2) configuration:
Platform:
osname=freebsd
osvers=11.1-release-p1
archname=amd64-freebsd-thread-multi
uname='freebsd 111amd64-default-supernews-job-01 11.1-release-p1 freebsd 11.1-release-p1 amd64 '
config_args='-sde -Dprefix=/usr/local -Dlibperl=libperl.so.5.26.2 -Darchlib=/usr/local/lib/perl5/5.26/mach -Dprivlib=/usr/local/lib/perl5/5.26 -Dman3dir=/usr/local/lib/perl5/5.26/perl/man/man3 -Dman1dir=/usr/local/lib/perl5/5.26/perl/man/man1 -Dsitearch=/usr/local/lib/perl5/site_perl/mach/5.26 -Dsitelib=/usr/local/lib/perl5/site_perl -Dscriptdir=/usr/local/bin -Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3 -Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dusenm=n -Dcc=cc -Duseshrplib -Dinc_version_list=none -Dcf_by=perl -Dcf_email=perl@FreeBSD.org -Dcf_time=Sat Apr 14 11:27:49 UTC 2018 -Alddlflags=-L/wrkdirs/usr/ports/lang/perl5.26/work/perl-5.26.2 -L/usr/local/lib/perl5/5.26/mach/CORE -lperl -Dshrpldflags=$(LDDLFLAGS:N-L/wrkdirs/usr/ports/lang/perl5.26/work/perl-5.26.2:N-L/usr/local/lib/perl5/5.26/mach/CORE:N-lperl) -Wl,-soname,$(LIBPERL:R) -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing -Dusedtrace -Ui_gdbm -Dusemultiplicity=y -Duse64bitint -Dusemymalloc=n -Dusesitecustomize -Dusethreads=y'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='4.2.1 Compatible FreeBSD Clang 4.0.0 (tags/RELEASE_400/final 297347)'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags ='-pthread -Wl,-E -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/lib /usr/local/lib /usr/bin/../lib/clang/4.0.0/lib /usr/lib
libs=-lpthread -lm -lcrypt -lutil
perllibs=-lpthread -lm -lcrypt -lutil
libc=
so=so
useshrplib=true
libperl=libperl.so.5.26.2
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags=' -Wl,-R/usr/local/lib/perl5/5.26/mach/CORE'
cccdlflags='-DPIC -fPIC'
lddlflags='-shared -L/usr/local/lib/perl5/5.26/mach/CORE -lperl -L/usr/local/lib -fstack-protector-strong'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
USE_SITECUSTOMIZE
Built under freebsd
@INC:
/usr/local/lib/perl5/site_perl/mach/5.26
/usr/local/lib/perl5/site_perl
/usr/local/lib/perl5/5.26/mach
/usr/local/lib/perl5/5.26
That USE_SITECUSTOMIZE is key to our cunning plan.
Now, back to that INSTALL post.
Where does it go?
My next problem was: where is $sitelibexp? That is the question I put to #perl on IRC and haarg gave me this command:
$ perl -V:sitelibexp sitelibexp='/usr/local/lib/perl5/site_perl';
OK, good, so I need /usr/local/lib/perl5/site_perl/sitecustomize.pl. Based on something I found while researching this topic, I tried this:
$ cat /usr/local/lib/perl5/site_perl/sitecustomize.pl push @INC, "/usr/websites/freshports.org/scripts";
Now look at @INC:
$ perl -E 'say for @INC' /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26 /usr/websites/freshports.org/scripts
SCORE!
The FreshPorts scripts are happy for now. I’ll upgrade to 5.26 after the old server has been running it carefree for a few days.
