Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve installation docs and include running Pelican server with docker image #904

Merged
merged 5 commits into from
Mar 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/pages/_meta.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"index": "About Pelican",
"install": "Install",
"install": "Install Pelican",
"parameters": "Configuration",
"client-usage": "Client Usage",
"serving_an_origin":"Serving an Origin",
Expand Down
98 changes: 26 additions & 72 deletions docs/pages/install.mdx
Original file line number Diff line number Diff line change
@@ -1,96 +1,50 @@
import DownloadsComponent from "/components/DownloadsComponent";

# Installing the Pelican Platform
# Install Pelican

This document explains how a user can download and install Pelican client or Pelican server components.
This document lists the operating system requirements to install Pelican and explains how a user can download Pelican client or Pelican server components.

## Installation Steps:
> Pelican also has Docker images for users who want to serve **Pelican servers**. Refer to [Run Pelican Server using Docker image](./install/docker.mdx) for instructions.

## Supported Operating Sytems

Pelican supports the following operating systems:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be careful in making this claim. While we can technically run the pelican binary on Windows, most of the functionality is turned of or compiled out. It's not as bad with MacOS, but we're still largely untested there as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think is is for windows only or the claim that we support the OS? If it's former, then I can add a note to Windows item suggesting that we only support client functionality. This would be the same thing for macOS where I don't think xrootd works well with mac and people should be required to build xrootd from source

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I haven't even tested the client on Windows and I've not examined the extent of our windows tests. Does the client fully work there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've bugged @joereuss12 to run the .exe program we generated and see if that works


* [Red Hat Enterprise Linux (CentOS, Fedora, Alma, Rocky, or openSUSE)](./install/rhel.mdx)
* [Debian or Ubuntu](./install/debian.mdx)
* [Alpine Linux](./install/alpine.mdx)
* [macOS](./install/macos.mdx) [*]
* Windows [*]

> Note that for macOS and Windows, Pelican only supports the _client_ functionality but not the _server_ functionality. To launch a Pelican server on macOS or Windows, use [Pelican Docker image](./install/docker.mdx) instead.

> Note that installation of Pelican on other operating systems is possible, but not recommended or supported.

## Download Pelican Binary

### 1. Install the Pelican Platform Binary
Download the proper binary for the system you are running on and select which version you would like to install. If you do not know which binary to install, below is a chart that may help:

<DownloadsComponent />

##### RPM, APK, or DEB?
###### RPM (Red Hat Package Manager):
### RPM, APK, or DEB?

#### RPM (Red Hat Package Manager):
You want to install a `.rpm` package if you are using a Red Hat-based Linux distribution system such as: Red Hat Enterprise Linux, CentOS, Fedora, or openSUSE.

###### APK (Alpine Package Keeper):
#### APK (Alpine Package Keeper):
You want to install a `.apk` package if you are using an Alpine Linux system.

###### DEB (Debian Package):
#### DEB (Debian Package):
You want to install a `.deb` package if you are using a Linux distribution system such as: Debian, Ubuntu, or something similar.

##### What about tar and zip?
#### What about tar and zip?
If you want a more manual setup, you can download the `.tar.gz` or `.zip` files and extract the binary where you need it. However, the above packages are recommended for more inexperienced users.

##### What version should I download?
### What version should I download?
Our versions are built like so:
For example: 7.1.2
- 7 represents the major version release
- 1 represents feature releases
- 2 represents a bug fix/patch release

We recommend you to use the latest feature or major release version so it includes the latest bug fixes and features.

### 2. Extract the Binary
Once the package has finished downloading, place it in your workspace

##### RPM:
To install RPM packages, you can use the dnf or yum package manager. Just make sure you have sudo access if you are not root:
```console
sudo yum install pelican-7.1.2-1.x86_64.rpm
```
Replace 'yum' with 'dnf' for dnf install

##### DEB:
To install Debian packages, you can use apt or dpkg package manager:
```console
sudo dpkg -i pelican-7.1.2-1_amd64.deb
sudo apt-get install -f
```
Or
```console
sudo apt install ./pelican-7.1.2-1_amd64.deb
```

##### APK:
To install APK packages on Alpine Linux, you can use the apk package manager:
```console
apk add pelican_amd64.apk
```

##### Tarballs*:
To install `.tar.gz` packages, you can extract with tar:
```console
tar -xzf pelican_Darwin_x86_64.tar.gz
```

##### Zip*:
To install `.zip` packages, you can use [7zip](https://www.7-zip.org/) or other programs to unzip your file. Simply right click on the `.zip` file and extract all contents to a directory of your choice


>**\*Note:** If you install a tarball or zipfile, you don't actually *install* Pelican, you just extract the binary. It's covered below how you can still use Pelican with the binary, but if you would like to have similar functionality as the other packages, you need to add Pelican to your PATH manually.

### 3. Test Functionality of the Pelican Platform
##### For rpm, deb, and apk
If you downloaded this way, Pelican should automatically be added to the path. You can check this by:
```console
which pelican
```
and make sure you get an output. You can then check functionality by running a simple **object copy** command:
```console
pelican -f osg-htc.org object copy /ospool/uc-shared/public/OSG-Staff/validation/test.txt .
testfile.txt 36.00 b / 36.00 b [=============================================================================================] Done!
```

##### For tarballs and zip, using the binary itself:
Once extracted, make sure you are in the same directory as the **Pelican** executable. To test if everything works, we can do a simple **object copy** command:

```console
./pelican -f osg-htc.org object copy /ospool/uc-shared/public/OSG-Staff/validation/test.txt .
testfile.txt 36.00 b / 36.00 b [=============================================================================================] Done!
```

You should now notice a file **testfile.txt** now in your directory. Congrats on making your first Pelican Object Copy!
The download section above always shows the latest released version of Pelican and we recommend you to use the latest feature or major release version so it includes the latest bug fixes and features. If you would like to download a previous version, please refer to our [GitHub release page](https://github.com/PelicanPlatform/pelican/releases).
7 changes: 7 additions & 0 deletions docs/pages/install/_meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"rhel": "CentOS, Fedora, or openSUSE",
"debian": "Debian and Ubuntu",
"alpine": "Alpine Linux",
"macos": "macOS",
"dockder": "Pelican Docker Image"
}
51 changes: 51 additions & 0 deletions docs/pages/install/alpine.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Install Pelican on Alpine

This document explains how to install Pelican on a Alpine Linux operating system.

## Install Pelican APK

You can install Pelican by downloading an `.apk` package or by downloading a binary `.tar.gz` file. You only need to follow one section below to install.

### Install the Pelican using an APK pacakge

1. Navigate to [Pelican download page](../install.mdx) and select the Pelican APK you want to install.

2. In **Operating System** section, select **Linux**. In **Architechtures** section, select **AMD64** (Intel/AMD) or **ARM64** (ARM) depending on the architecture of your machine.

3. In the list of download candidates, copy the link to `pelican_x.x.x_p1_aarch64.apk` if you select **AARCH64** (ARM), or `pelican_x.x.x_p1_x86_64.apk` if you select **X84_64** (Intel/AMD). Where `x.x.x` is the version number.

4. Change the following command with the link to the binary you copied in the previous step and run the command

```bash
wget <replace-with-the-link-you-copied>
sudo apk add --allow-untrusted pelican_7.5.8_p1_aarch64.apk
```

Note that you need to replace `pelican_7.5.8_p1_aarch64.apk` with the file name you downloaded to your current directory.

Example to install Pelican `v7.5.8` APK for an Intel/AMD machine:

```bash
wget https://github.com/PelicanPlatform/pelican/releases/download/v7.5.8/pelican_7.5.8_p1_x86_64.apk
sudo apk add --allow-untrusted pelican_7.5.8_p1_x86_64.apk
```

### Install Pelican as a standalone binary

Refer to the [previous section](./rhel.mdx#install-pelican-as-a-standalone-binary) on how to install Pelican as a standalone binary.

## Uninstall on Alpine

To uninstall Pelican, run the following command

```bash
sudo apk del pelican
```

## Verify Pelican is installed

Refer to the [previous section](./rhel.mdx#verify-pelican-is-installed) on how to verify that Pelican is successfully installed.

## Next steps

If you want to use Pelican client to download or upload files, refer to [Client Usage](../client-usage.mdx) to get started. If you want to serve a Pelican server, refer to [Serve a Pelican Origin](../serving_an_origin.mdx) and [Serve a Pelican Federation](../serving_a_federation.mdx).
51 changes: 51 additions & 0 deletions docs/pages/install/debian.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Install Pelican on Debian or Ubuntu

This document explains how to install Pelican on a Debian or Ubuntu Linux operating system.

## Install Pelican DEB

You can install Pelican by downloading a `.deb` package or by downloading a binary `.tar.gz` file. You only need to follow one section below to install.

### Install the Pelican using a DEB pacakge

1. Navigate to [Pelican download page](../install.mdx) and select the Pelican DEB you want to install.

2. In **Operating System** section, select **Linux**. In **Architechtures** section, select **AMD64** (Intel/AMD) or **ARM64** (ARM) depending on the architecture of your machine.

3. In the list of download candidates, copy the link to `pelican-x.x.x-1_arm64.deb` if you select **ARM64** (ARM), or `pelican-x.x.x-1.amd64.deb` if you select **AMD64** (Intel/AMD). Where `x.x.x` is the version number.

4. Change the following command with the link to the binary you copied in the previous step and run the command

```bash
wget <replace-with-the-link-you-copied>
sudo dpkg -i pelican_7.5.8-1_arm64.deb
```

Note that you need to replace `pelican_7.5.8-1_arm64.deb` with `pelican_7.5.8-1_amd64.deb` if you are running an Intel/AMD machine.

Example to install Pelican `v7.5.8` DEB package for an Intel/AMD machine:

```bash
wget https://github.com/PelicanPlatform/pelican/releases/download/v7.5.8/pelican_7.5.8-1_amd64.deb
sudo dpkg -i pelican_7.5.8-1_amd64.deb
```

### Install Pelican as a standalone binary

Refer to the [previous section](./rhel.mdx#install-pelican-as-a-standalone-binary) on how to install Pelican as a standalone binary.

## Uninstall on Debian or Ubuntu

To uninstall Pelican, run the following command

```bash
sudo apt-get remove pelican
```

## Verify Pelican is installed

Refer to the [previous section](./rhel.mdx#verify-pelican-is-installed) on how to verify that Pelican is successfully installed.

## Next steps

If you want to use Pelican client to download or upload files, refer to [Client Usage](../client-usage.mdx) to get started. If you want to serve a Pelican server, refer to [Serve a Pelican Origin](../serving_an_origin.mdx) and [Serve a Pelican Federation](../serving_a_federation.mdx).
84 changes: 84 additions & 0 deletions docs/pages/install/docker.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Run Pelican Server using Docker image

This document explains how to run a Pelican server using Pelican docker image. If you are installing Pelican to use its client functionalities (download or upload an object), refer to previous sections to download and install a Pelican binary instead.

## Before starting

Pelican builds separate images for each Pelican server components, e.g. origin, cache, director, and registry. Depending on which Pelican server you want to run, you need to select a different Docker image from the list below.

* Pelican origin server: `hub.opensciencegrid.org/pelican_platform/origin:latest`
* Pelican cache server: `hub.opensciencegrid.org/pelican_platform/cache:latest`
* Pelican director server: `hub.opensciencegrid.org/pelican_platform/director:latest`
* Pelican registry server: `hub.opensciencegrid.org/pelican_platform/registry:latest`

## Run Pelican server via Docker CLI

This section shows you how to run various Pelican server images using the Docker CLI. If you haven't installed Docker engine, follow the [documentation from Docker](https://docs.docker.com/get-docker/) to install the Docker engine.


### Run Pelican origin server

To run the latest pelican origin server, run the following command:

```bash
docker run -it -p 8444:8444 -p 8443:8443 -v /path/to/your/data/:/tmp/pelican --name=pelican-origin hub.opensciencegrid.org/pelican_platform/origin:latest -v /tmp/pelican:/foo/bar
```

Where:

* `docker run` is a Docker CLI command that runs a new container from an image
* `-it` (`--interactive --tty`) runs the container in interactive mode and uses a tty terminal
* `-p <host-port>:<container-port>` (`--publish`) publishes a container’s port(s) to the host, allowing you to reach the container’s port via a host port. In this case, we can reach the container’s port `8444` via the host’s port `8444`. Note that the web engine of the Pelican server runs on port `8444` by default, and the file transfer endpoint of the Pelican server runs on port `8443` by default.
* `-v <host-location>:<container-location>` (`--volume`) binds mount a volume from the host location(s) to the containter's location(s). This allow you to share files in your host machine to the container. In this case, we bind `/path/to/your/data/` on your host machine to `/tmp/pelican` in the container. You need to replace `/path/to/your/data/` to the directory where your data to publish is located.
* `--name` assigns a logical name to the container (e.g. pelican-origin). This allows you to refer to the container by name instead of by ID.
* `hub.opensciencegrid.org/pelican_platform/origin:latest` is the image to run
* `-v /tmp/pelican:/foo/bar` is the Pelican argument to bind `/tmp/pelican` directory in the container as namespace `/foo/bar` in Pelican. You need to change `/foo/bar` to a meaningful path that can represent your data, e.g. `/chtc/public-data`. You may pass additional arguments to Pelican server by appending them after this argument.

### Run Pelican cache server

To run the latest pelican cache server, run the following command:

```bash
docker run -it -p 8444:8444 -p 8442:8442 --name=pelican-cache hub.opensciencegrid.org/pelican_platform/cache:latest
```

> Note that Pelican docker image currently does not support binding a directory on your host machine as the directory for Pelican cache to store cache files, i.e. using `-v /host/machine:/run/pelican/cache/location`. We will have an update once we officialy support this flow.

### Run Pelican director server

To run the latest pelican director server, run the following command:

```bash
docker run -it -p 8444:8444 --name=pelican-director hub.opensciencegrid.org/pelican_platform/director:latest
```

> Note that to successfully run a Pelican director server, additional configuration is required. Follow [Serve a Pelican Federation](../serving_a_federation.mdx) for instructions.

### Run Pelican registry server

To run the latest pelican registry server, run the following command:

```bash
docker run -it -p 8444:8444 --name=pelican-registry hub.opensciencegrid.org/pelican_platform/registry:latest
```

> Note that to successfully run a Pelican registry server, additional configuration is required. Follow [Serve a Pelican Federation](../serving_a_federation.mdx) for instructions.


## Stop Pelican container

To stop the Pelican container, run the following command:

```bash
# The `docker ps` command shows the processes running in Docker
docker ps

# This will display a list of containers that looks like the following:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0be1a304b5d7 hub.opensciencegrid.org/pelican_platform/director:latest "/bin/sh" 1 hour ago Up 1 hour 0.0.0.0:8444->8444/tcp pelican-director

# To stop the pelican container run the command
# docker stop <container-ID> or use
# docker stop <container-name>, which is `pelican-director` as previously defined
docker stop pelican-director
```
37 changes: 37 additions & 0 deletions docs/pages/install/macos.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Install Pelican on macOS

This document explains how to install Pelican on macOS.

### Install Pelican as a standalone binary

1. Navigate to [Pelican download page](../install.mdx) and select the Pelican macOS binary you want to install.

2. In **Operating System** section, select **macOS**. In **Architechtures** section, select **X84_64** (Intel) or **ARM64** (ARM) depending on the architecture of your machine.

3. In the list of candidates, copy the link to `pelican_Darwin_arm64.tar.gz` if you select **ARM64** (ARM), or `pelican_Darwin_x86_64.tar.gz` if you select **X84_64** (Intel).

4. Change the following command with the link to the binary you copied in the previous step and run the command

```bash
wget <replace-with-the-link-you-copied>
tar -zxvf pelican_Darwin_arm64.tar.gz // ARM user
```

Note that you need to replace `pelican_Darwin_arm64.tar.gz` with `pelican_Darwin_x86_64.tar.gz` if you are running an Intel machine.

Example to install Pelican binary for an Intel machine:

```bash
curl -O https://github.com/PelicanPlatform/pelican/releases/download/v7.5.8/pelican_Darwin_x86_64.tar.gz
tar -zxvf pelican_Darwin_x86_64.tar.gz
```

5. At this step, you don't actually install Pelican, you just extract the binary. It is recommended that you add Pelican binary to your `PATH` environment variable to allow Pelican to be called from your command line.

## Verify Pelican is installed

Refer to the [previous section](./rhel.mdx#verify-pelican-is-installed) on how to verify that Pelican is successfully installed.

## Next steps

If you want to use Pelican client to download or upload files, refer to [Client Usage](../client-usage.mdx) to get started. If you want to serve a Pelican server, refer to [Serve a Pelican Origin](../serving_an_origin.mdx) and [Serve a Pelican Federation](../serving_a_federation.mdx).
Loading
Loading