Apr 222018
 

This post is all about creating technical debt. If you accept that, go for it. I’m avoiding porting my FreshPorts scripts into SITEPERL. Why? I’ll migrate them to SITEPERL after BSDCan & PGCon. Right now, I need to get the servers upgraded from Perl 5.24 to Perl 5.26, because 5.24 is deprecated.

FreshPorts uses Perl for processing incoming commits and for various administrative backend tasks. Everything on the front end (website) is PHP and HTML. Perl is great at text processing, and text processing is at the heart of what drives FreshPorts.

Lately, my daily security run output FreeBSD emails have been mentioning:

perl5-5.24.4: Tag: expiration_date Value: 2019-05-09
perl5-5.24.4: Tag: deprecated Value: Support ends three years after .0 release, please upgrade to a more recent version of Perl

I’ve been using Perl 5.24 for a while. It is the default on my Poudriere builds.

I decided to upgrade the old FreshPorts server to Perl 5.26, which is now the default on FreeBSD, since 19 April 2018.

Immediately after upgrading, I started getting this problem:

Can't locate port.pm in @INC (you may need to install the port module) (@INC contains: /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26) at hourly_stats.pl line 10.
BEGIN failed--compilation aborted at hourly_stats.pl line 10.

Can't locate db_utils.pm in @INC (you may need to install the db_utils module) (@INC contains: /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26) at test-master-port.pl line 15.
BEGIN failed--compilation aborted at test-master-port.pl line 15.

Can't locate port.pm in @INC (you may need to install the port module) (@INC contains: /usr/local/lib/perl5/site_perl/mach/5.26 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.26/mach /usr/local/lib/perl5/5.26) at /usr/websites/freshports.org/scripts/daily_rendering_times.pl line 10.
BEGIN failed--compilation aborted at /usr/websites/freshports.org/scripts/daily_rendering_times.pl line 10.

Those are from three separate scripts.

Options

I started investigating and found a few options:

  1. I could alter all the perl scripts with

    use lib ‘path.to.files’

    But that hardcodes the location in the script. Less than ideal.
  2. I could set PERL5LIB in an environment variable.
  3. I could use -I when the scripts are invoked.

I decided, based on suggestions from mat_ on IRC to use the SITECUSTOMIZE option when building Perl.

Enabling SITECUSTOMIZE

I added this to the supernew-make.conf file for my poudriere build:

lang_perl5.26_SET+=SITECUSTOMIZE

Then I rebuilt Perl and installed it. Then I searched for perl SITECUSTOMIZE and found
the INSTALL docs for Perl 5.26. What I learned there: When enabled, this makes perl run $sitelibexp/sitecustomize.pl before anything else. This script can then be set up to add additional entries to @INC.

Great. So what do I have now? This is how you see @INC for your perl:

$ perl -E 'map{say}@INC'
/usr/local/lib/perl5/site_perl/mach/5.26
/usr/local/lib/perl5/site_perl
/usr/local/lib/perl5/5.26/mach
/usr/local/lib/perl5/5.26

As ChoHag pointed out: ‘say for @INC’ is a simpler idiom than ‘map{say}@INC’. Which means this also works:

$ perl -E 'say for @INC'
/usr/local/lib/perl5/site_perl/mach/5.26
/usr/local/lib/perl5/site_perl
/usr/local/lib/perl5/5.26/mach
/usr/local/lib/perl5/5.26
/usr/websites/freshports.org/scripts

You can also use this command to see various environment variables related to Perl installed on your system:

$ env -i perl -V
Summary of my perl5 (revision 5 version 26 subversion 2) configuration:
   
  Platform:
    osname=freebsd
    osvers=11.1-release-p1
    archname=amd64-freebsd-thread-multi
    uname='freebsd 111amd64-default-supernews-job-01 11.1-release-p1 freebsd 11.1-release-p1 amd64 '
    config_args='-sde -Dprefix=/usr/local -Dlibperl=libperl.so.5.26.2 -Darchlib=/usr/local/lib/perl5/5.26/mach -Dprivlib=/usr/local/lib/perl5/5.26 -Dman3dir=/usr/local/lib/perl5/5.26/perl/man/man3 -Dman1dir=/usr/local/lib/perl5/5.26/perl/man/man1 -Dsitearch=/usr/local/lib/perl5/site_perl/mach/5.26 -Dsitelib=/usr/local/lib/perl5/site_perl -Dscriptdir=/usr/local/bin -Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3 -Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dusenm=n -Dcc=cc -Duseshrplib -Dinc_version_list=none -Dcf_by=perl -Dcf_email=perl@FreeBSD.org -Dcf_time=Sat Apr 14 11:27:49 UTC 2018 -Alddlflags=-L/wrkdirs/usr/ports/lang/perl5.26/work/perl-5.26.2 -L/usr/local/lib/perl5/5.26/mach/CORE -lperl -Dshrpldflags=$(LDDLFLAGS:N-L/wrkdirs/usr/ports/lang/perl5.26/work/perl-5.26.2:N-L/usr/local/lib/perl5/5.26/mach/CORE:N-lperl) -Wl,-soname,$(LIBPERL:R) -Doptimize=-O2 -pipe  -fstack-protector -fno-strict-aliasing -Dusedtrace -Ui_gdbm -Dusemultiplicity=y -Duse64bitint -Dusemymalloc=n -Dusesitecustomize -Dusethreads=y'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
    optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible FreeBSD Clang 4.0.0 (tags/RELEASE_400/final 297347)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib /usr/bin/../lib/clang/4.0.0/lib /usr/lib
    libs=-lpthread -lm -lcrypt -lutil
    perllibs=-lpthread -lm -lcrypt -lutil
    libc=
    so=so
    useshrplib=true
    libperl=libperl.so.5.26.2
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='  -Wl,-R/usr/local/lib/perl5/5.26/mach/CORE'
    cccdlflags='-DPIC -fPIC'
    lddlflags='-shared  -L/usr/local/lib/perl5/5.26/mach/CORE -lperl -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
    USE_SITECUSTOMIZE
  Built under freebsd
  @INC:
    /usr/local/lib/perl5/site_perl/mach/5.26
    /usr/local/lib/perl5/site_perl
    /usr/local/lib/perl5/5.26/mach
    /usr/local/lib/perl5/5.26

That USE_SITECUSTOMIZE is key to our cunning plan.

Now, back to that INSTALL post.

Where does it go?

My next problem was: where is $sitelibexp? That is the question I put to #perl on IRC and haarg gave me this command:

$ perl -V:sitelibexp
sitelibexp='/usr/local/lib/perl5/site_perl';

OK, good, so I need /usr/local/lib/perl5/site_perl/sitecustomize.pl. Based on something I found while researching this topic, I tried this:

$ cat /usr/local/lib/perl5/site_perl/sitecustomize.pl 
push @INC, "/usr/websites/freshports.org/scripts";

Now look at @INC:

$ perl -E 'say for @INC'
/usr/local/lib/perl5/site_perl/mach/5.26
/usr/local/lib/perl5/site_perl
/usr/local/lib/perl5/5.26/mach
/usr/local/lib/perl5/5.26
/usr/websites/freshports.org/scripts

SCORE!

The FreshPorts scripts are happy for now. I’ll upgrade to 5.26 after the old server has been running it carefree for a few days.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive