Debian packaging

[vjrj]

How to use the ipt debian package:

You can use the debian package of this PR adding this repo:
https://github.com/AtlasOfLivingAustralia/documentation/wiki/Testing-Debian-Packaging#using-demo-repository
and:

apt install ipt

Also you can download the packages manually and use dpkg -i some-ipt.deb:
https://apt.gbif.es/pool/main/i/ipt/

When you install the package it ask for the datadir:

In previous versions to 2.7.3 you have to configure the datadir manually and it gives you some extra manual steps to continue (not needed in recent versions). Here are the old manual steps:

You can use /var/lib/ipt as datadir.location
Tomcat is sandboxed by systemd and only has write access to some
directories (see /usr/share/doc/tomcat9/README.Debian):

To give access to your IPT data directory, settings have to be overridden.
This is done by creating an override.conf file in /etc/systemd/system/tomcat9.service.d/ containing:

[Service]
ReadWritePaths=/path/to/the/ipt-data-directory/

The service has to be restarted afterward with:

systemctl daemon-reload
systemctl restart tomcat9 

This is also commented in the IPT FAQ:
https://ipt.gbif.org/manual/en/ipt/2.5/faq#sandboxing

Original text of this PR

@Meshack007 from Tanzania dev team, asked me last Friday for help to install the IPT.

Instead to install it manually I prefered to use:
https://github.com/living-atlases/living-atlases-debianizer
to generate a debian package together, as we were using LXC ubuntu images as containers (and docker cannot run there).

Each debian directory file is commented pointing to some part of Debian documentation or to some man page. Also there is a README with more info about the package.

A summary of how to build the debian package:

sudo apt install debhelper
sudo apt install devscripts
debuild -us -uc -b

We used tomcat8 as dependency because I didn't find jetty-runner or similar already packaged in debian/ubuntu. So it doesn't run like a independent service right now maybe in the future as I did with cas5 debian testing package:
https://github.com/living-atlases/ala-cas-5/blob/debian-package/debian/ala-cas-5.service

I also included in our test apt repository:
https://github.com/AtlasOfLivingAustralia/documentation/wiki/Testing-Debian-Packaging#using-demo-repository
so you can test the install via apt install ipt using this repository.

Some suggestions about apt repositories:
https://github.com/AtlasOfLivingAustralia/documentation/wiki/Testing-Debian-Packaging#create-your-own-apt-repository

[MattBlissett]

Thanks @vjrj

If anyone else would like Debian packages, please comment on this issue. (please comment rather than 👍 and write which institution is interested.)

I won't merge it until there's some demand. We only have ~8 servers polling the CentOS/RedHat repository, and maintaining (including testing) a Debian package is more difficult for us, since we mostly use CentOS at GBIF.

I also wrote the CentOS build to run within a Docker container, for portability and reliability in any environment. Just debuild -us -uc -b didn't produce a .deb for me, on my Kubuntu desktop. (Would it be simpler to simply retrieve the built WAR from the Nexus repository? I realize that wouldn't satisfy the Debian package repository's requirements, but I don't know if you were aiming for that.)

[vjrj]

About the build, I didn't included the war generation because I usually add a extra deb build step after a jenkins war build job. Also because I'm not sure of your build and release procedure. So please build the war before the debuild command [1].

I recently asked in our small LA community about RedHat/related usage and I didn't find any LA portal using it.

As I mentioned, in Tanzania docker was not an option in that moment.

To package fat wars is common in non official debian packages when their dependencies are not packaged (I can remember openfire chat server package [2], for instance). java is not so easy to package compared to python libs IMHO.

Thanks @MattBlissett

[1] In the case of the la-pipelines debian package, I did this war build conditional:
https://github.com/gbif/pipelines/blob/ala-dev/livingatlas/debian/rules#L23
and I have a extra jenkins job that build it. The packages:
https://apt.gbif.es/pool/main/l/la-pipelines/
[2] see the rules/makefile https://github.com/igniterealtime/Openfire/blob/master/build/debian/rules

[vjrj]

I updated this PR to master. Now also do the build and I configured it to use with:
https://jenkins-debian-glue.org/
because I didn't have a job to update the repository.

[vjrj]

I've just updated this package to use tomcat9. The 2.4.2 for tomcat9 can be installed with:

apt install ipt=2.4.2+0~20210714095243.36~1.gbp6bc430 

Tomcat is sandboxed by systemd in Debian and only has write access to some directories (see /usr/share/doc/tomcat9/README.Debian). To give access to your IPT data directory, settings have to be overridden.

This is done by creating an override.conf file in /etc/systemd/system/tomcat9.service.d/ containing:

[Service]
ReadWritePaths=/path/to/the/ipt-data-directory/

The service has to be restarted afterward with:

systemctl daemon-reload
systemctl restart tomcat9

This is also useful for ipt manual installations in debian/ubuntu/etc using recent tomcat versions.

[vjrj]

I've just updated to 2.5.0 running for now in https://ipt-demo.gbif.es/

So apt install ipt installs 2.5.0 now from apt.gbif.es.

[vjrj]

I've just rebased this PR and build/published 2.5.1 debian package. It's running in https://ipt-demo.gbif.es/

I'll update our https://ipt.gbif.es that is running 2.5.0.

So far I detected in 2.5.0:

• an error in health check (javaVersion is null for some reason, I didn't find yet the source of the problem).
• the login redirects to https://ipt.gbif.es/ipt instead of https://ipt.gbif.es/ This is maybe a proxy issue (we use nginx), but I didn't find a fix yet.

[MattBlissett]

Thanks for keeping this up-to-date.

Is the public URL / baseURL set correctly, and is Nginx changing the path at all? Apache uses the ProxyPathReverse parameter when adjustments are required: https://ipt.gbif.org/manual/en/ipt/2.5/installation#opening-the-ipt-to-the-internet

[vjrj]

Thanks for your help!

Our conf:

ipt.baseURL=https\://ipt-demo.gbif.es/

and I checked that page from the IPT manual for setting up our proxies but the nginx equivalent conf
https://www.nginx.com/resources/wiki/start/topics/examples/likeapache/
did not work:

    location / {
        proxy_pass http://1.2.3.4:8080/ipt/;
        proxy_set_header Host $host;

        proxy_cookie_path ~*^/.* /;
        proxy_set_header X-Forwarded-Host $host:$server_port;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    }

I have to do more tests.

[eugengo]

For nginx users and login redirect error, because of session.cookie.path:

Changes in /var/lib/tomcat9/conf/server.xml

<Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443"
               sessionCookiePath="/" />

and nginx site.conf file:

location / {
        proxy_set_header X-Forwarded-Host $host:$server_port;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        
        proxy_cookie_path /ipt/ "/; secure";

        proxy_pass http://127.0.0.1:8080/ipt/;
}

Restart tomcat

Also you need to check permissions on /var/lib/tomcat9/webapps/ipt/
nginx needs +x bit on directories and files

[vjrj]

I've just rebased this PR and published the resulting 2.5.2 debian package. It's running in https://ipt-demo.gbif.es/

Thanks!

[raffael-mnhn]

https://www.mnhn.lu is interested in a debian package.

[vjrj]

Hi @raffael-mnhn right now you can use it adding this repo:
https://github.com/AtlasOfLivingAustralia/documentation/wiki/Testing-Debian-Packaging#using-demo-repository
and:

apt install ipt

Also you can download the packages manually and use dpkg -i some-ipt.deb:
https://apt.gbif.es/pool/main/i/ipt/

[raffael-mnhn]

@vjrj it seems that there is only a package for bionic available. I'm on ubuntu focal. Do you expect it to work nonetheless? I'm going to give it a try and report back.

[raffael-mnhn]

@vjrj The following command also fails
# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F697D8D2ADB9E24A

I noticed that the key cannot be found on the web service of the keyserver aswell. Maybe it's out of date and was removed.

I just added the [trusted=yes] option to the repo configuration in order to make it work without checking the signature.

[vjrj]

Hi @raffael-mnhn about 20.04 we used the package in 20.04 too. Sorry this was just a testing apt repo for a debian proposal that we are using it meanwhile.

About using the last version, I installed it in https://ipt.gbif.es but I discover this issue #1691 (see a fix in the comments). In ipt.gbif.es we are using a local created deb package with this workaround as we were using java 11 in that server. The apt.gbif.es deb package does not include this workaround right now.

So you have two options,

• try to use java 8 with the apt.gbif.es published debian package
• or to build a deb package with that workaround that works in java 11

About the gpg key you can found it also here:
https://keyserver.ubuntu.com/pks/lookup?op=hget&search=7524d3b383016eab0ee47bfe253a51f7

[raffael-mnhn]

For those who wonder how to achieve the first solution proposed by @vjrj :
# apt install openjdk-8-jre-headless
# update-alternatives --config java choose version 8
# apt purge openjdk-11-jre-headless to get rid of 11 on the system

[vjrj]

Updated to 2.5.4 (security release) and running in ipt.gbif.es.

Thanks @MattBlissett

[raffael-mnhn]

@vjrj there is another vulnerability in log4j 2.5.4, cf: https://www.lunasec.io/docs/blog/log4j-zero-day-update-on-cve-2021-45046/ and https://www.lunasec.io/docs/blog/log4j-zero-day-mitigation-guide/

[MattBlissett]

The other vulnerability doesn't affect the IPT, as it doesn't use the MDC/ThreadContext feature, and doesn't include the relevant patterns in the logging formats.

[vjrj]

The 3.0.1 debian version is available for install. Just update it with:

apt update
apt install ipt

[vjrj]

The 3.0.2 debian version is available for install. I'll publish newer versions silently.