-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve installation docs and include running Pelican server with docker image #904
Merged
Merged
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
966b824
Improve installation docs
haoming29 df9b30b
Update Pelican cache in docker container
haoming29 6aa35d7
Update links to be relative and copy errors
haoming29 b40af63
Explain on the level of support we have for windows and macos
haoming29 ddd68d9
Update install doc to reflect change in running docker image
haoming29 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,96 +1,48 @@ | ||
import DownloadsComponent from "/components/DownloadsComponent"; | ||
|
||
# Installing the Pelican Platform | ||
# Install Pelican | ||
|
||
This document explains how a user can download and install Pelican client or Pelican server components. | ||
This document lists the operating system requirements to install Pelican and explains how a user can download Pelican client or Pelican server components. | ||
|
||
## Installation Steps: | ||
> Pelican also has Docker images for users who want to serve **Pelican servers**. Refer to [Run Pelican Server using Docker image](./install/docker.mdx) for instructions. | ||
|
||
## Supported Operating Sytems | ||
|
||
Pelican supports the following operating systems: | ||
|
||
* [Red Hat, RHEL, CentOS, Fedora, or openSUSE](./install/rhel.mdx) | ||
* [Debian or Ubuntu](./install/debian.mdx) | ||
* [Alpine Linux](./install/alpine.mdx) | ||
* [macOS](./install/macos.mdx) | ||
* Windows | ||
|
||
> Note that installation of Pelican on other operating systems is possible, but not recommended or supported. | ||
|
||
## Download Pelican Binary | ||
|
||
### 1. Install the Pelican Platform Binary | ||
Download the proper binary for the system you are running on and select which version you would like to install. If you do not know which binary to install, below is a chart that may help: | ||
|
||
<DownloadsComponent /> | ||
|
||
##### RPM, APK, or DEB? | ||
###### RPM (Red Hat Package Manager): | ||
### RPM, APK, or DEB? | ||
|
||
#### RPM (Red Hat Package Manager): | ||
You want to install a `.rpm` package if you are using a Red Hat-based Linux distribution system such as: Red Hat Enterprise Linux, CentOS, Fedora, or openSUSE. | ||
|
||
###### APK (Alpine Package Keeper): | ||
#### APK (Alpine Package Keeper): | ||
You want to install a `.apk` package if you are using an Alpine Linux system. | ||
|
||
###### DEB (Debian Package): | ||
#### DEB (Debian Package): | ||
You want to install a `.deb` package if you are using a Linux distribution system such as: Debian, Ubuntu, or something similar. | ||
|
||
##### What about tar and zip? | ||
#### What about tar and zip? | ||
If you want a more manual setup, you can download the `.tar.gz` or `.zip` files and extract the binary where you need it. However, the above packages are recommended for more inexperienced users. | ||
|
||
##### What version should I download? | ||
### What version should I download? | ||
Our versions are built like so: | ||
For example: 7.1.2 | ||
- 7 represents the major version release | ||
- 1 represents feature releases | ||
- 2 represents a bug fix/patch release | ||
|
||
We recommend you to use the latest feature or major release version so it includes the latest bug fixes and features. | ||
|
||
### 2. Extract the Binary | ||
Once the package has finished downloading, place it in your workspace | ||
|
||
##### RPM: | ||
To install RPM packages, you can use the dnf or yum package manager. Just make sure you have sudo access if you are not root: | ||
```console | ||
sudo yum install pelican-7.1.2-1.x86_64.rpm | ||
``` | ||
Replace 'yum' with 'dnf' for dnf install | ||
|
||
##### DEB: | ||
To install Debian packages, you can use apt or dpkg package manager: | ||
```console | ||
sudo dpkg -i pelican-7.1.2-1_amd64.deb | ||
sudo apt-get install -f | ||
``` | ||
Or | ||
```console | ||
sudo apt install ./pelican-7.1.2-1_amd64.deb | ||
``` | ||
|
||
##### APK: | ||
To install APK packages on Alpine Linux, you can use the apk package manager: | ||
```console | ||
apk add pelican_amd64.apk | ||
``` | ||
|
||
##### Tarballs*: | ||
To install `.tar.gz` packages, you can extract with tar: | ||
```console | ||
tar -xzf pelican_Darwin_x86_64.tar.gz | ||
``` | ||
|
||
##### Zip*: | ||
To install `.zip` packages, you can use [7zip](https://www.7-zip.org/) or other programs to unzip your file. Simply right click on the `.zip` file and extract all contents to a directory of your choice | ||
|
||
|
||
>**\*Note:** If you install a tarball or zipfile, you don't actually *install* Pelican, you just extract the binary. It's covered below how you can still use Pelican with the binary, but if you would like to have similar functionality as the other packages, you need to add Pelican to your PATH manually. | ||
|
||
### 3. Test Functionality of the Pelican Platform | ||
##### For rpm, deb, and apk | ||
If you downloaded this way, Pelican should automatically be added to the path. You can check this by: | ||
```console | ||
which pelican | ||
``` | ||
and make sure you get an output. You can then check functionality by running a simple **object copy** command: | ||
```console | ||
pelican -f osg-htc.org object copy /ospool/uc-shared/public/OSG-Staff/validation/test.txt . | ||
testfile.txt 36.00 b / 36.00 b [=============================================================================================] Done! | ||
``` | ||
|
||
##### For tarballs and zip, using the binary itself: | ||
Once extracted, make sure you are in the same directory as the **Pelican** executable. To test if everything works, we can do a simple **object copy** command: | ||
|
||
```console | ||
./pelican -f osg-htc.org object copy /ospool/uc-shared/public/OSG-Staff/validation/test.txt . | ||
testfile.txt 36.00 b / 36.00 b [=============================================================================================] Done! | ||
``` | ||
|
||
You should now notice a file **testfile.txt** now in your directory. Congrats on making your first Pelican Object Copy! | ||
The download section above always shows the latest released version of Pelican and we recommend you to use the latest feature or major release version so it includes the latest bug fixes and features. If you would like to download a previous version, please refer to our [GitHub release page](https://github.com/PelicanPlatform/pelican/releases). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
{ | ||
"rhel": "CentOS, Fedora, or openSUSE", | ||
"debian": "Debian and Ubuntu", | ||
"alpine": "Alpine Linux", | ||
"macos": "macOS", | ||
"dockder": "Pelican Docker Image" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# Install Pelican on Alpine | ||
|
||
This document explains how to install Pelican on a Alpine Linux operating system. | ||
|
||
## Install Pelican APK | ||
|
||
You can install Pelican by downloading an `.apk` package or by downloading a binary `.tar.gz` file. You only need to follow one section below to install. | ||
|
||
### Install the Pelican using an APK pacakge | ||
|
||
1. Navigate to [Pelican download page](../install.mdx) and select the Pelican APK you want to install. | ||
|
||
2. In **Operating System** section, select **Linux**. In **Architechtures** section, select **AMD64** (Intel/AMD) or **ARM64** (ARM) depending on the architecture of your machine. | ||
|
||
3. In the list of download candidates, copy the link to `pelican_x.x.x_p1_aarch64.apk` if you select **AARCH64** (ARM), or `pelican_x.x.x_p1_x86_64.apk` if you select **X84_64** (Intel/AMD). Where `x.x.x` is the version number. | ||
|
||
4. Change the following command with the link to the binary you copied in the previous step and run the command | ||
|
||
```bash | ||
wget <replace-with-the-link-you-copied> | ||
sudo apk add --allow-untrusted pelican_7.5.8_p1_aarch64.apk | ||
``` | ||
|
||
Note that you need to replace `pelican_7.5.8_p1_aarch64.apk` with the file name you downloaded to your current directory. | ||
|
||
Example to install Pelican `v7.5.8` APK for an Intel/AMD machine: | ||
|
||
```bash | ||
wget https://github.com/PelicanPlatform/pelican/releases/download/v7.5.8/pelican_7.5.8_p1_x86_64.apk | ||
sudo apk add --allow-untrusted pelican_7.5.8_p1_x86_64.apk | ||
``` | ||
|
||
### Install Pelican as a standalone binary | ||
|
||
Refer to the [previous section](./rhel.mdx#install-pelican-as-a-standalone-binary) on how to install Pelican as a standalone binary. | ||
|
||
## Uninstall on Alpine | ||
|
||
To uninstall Pelican, run the following command | ||
|
||
```bash | ||
sudo apk del pelican | ||
``` | ||
|
||
## Verify Pelican is installed | ||
|
||
Refer to the [previous section](./rhel.mdx#verify-pelican-is-installed) on how to verify that Pelican is successfully installed. | ||
|
||
## Next steps | ||
|
||
If you want to use Pelican client to download or upload files, refer to [Client Usage](../client-usage.mdx) to get started. If you want to serve a Pelican server, refer to [Serve a Pelican Origin](../serving_an_origin.mdx) and [Serve a Pelican Federation](../serving_a_federation.mdx). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# Install Pelican on Debian or Ubuntu | ||
|
||
This document explains how to install Pelican on a Debian or Ubuntu Linux operating system. | ||
|
||
## Install Pelican DEB | ||
|
||
You can install Pelican by downloading a `.deb` package or by downloading a binary `.tar.gz` file. You only need to follow one section below to install. | ||
|
||
### Install the Pelican using a DEB pacakge | ||
|
||
1. Navigate to [Pelican download page](../install.mdx) and select the Pelican DEB you want to install. | ||
|
||
2. In **Operating System** section, select **Linux**. In **Architechtures** section, select **AMD64** (Intel/AMD) or **ARM64** (ARM) depending on the architecture of your machine. | ||
|
||
3. In the list of download candidates, copy the link to `pelican-x.x.x-1_arm64.deb` if you select **ARM64** (ARM), or `pelican-x.x.x-1.amd64.deb` if you select **AMD64** (Intel/AMD). Where `x.x.x` is the version number. | ||
|
||
4. Change the following command with the link to the binary you copied in the previous step and run the command | ||
|
||
```bash | ||
wget <replace-with-the-link-you-copied> | ||
sudo dpkg -i pelican_7.5.8-1_arm64.deb | ||
``` | ||
|
||
Note that you need to replace `pelican_7.5.8-1_arm64.deb` with `pelican_7.5.8-1_amd64.deb` if you are running an Intel/AMD machine. | ||
|
||
Example to install Pelican `v7.5.8` DEB package for an Intel/AMD machine: | ||
|
||
```bash | ||
wget https://github.com/PelicanPlatform/pelican/releases/download/v7.5.8/pelican_7.5.8-1_amd64.deb | ||
sudo dpkg -i pelican_7.5.8-1_amd64.deb | ||
``` | ||
|
||
### Install Pelican as a standalone binary | ||
|
||
Refer to the [previous section](./rhel.mdx#install-pelican-as-a-standalone-binary) on how to install Pelican as a standalone binary. | ||
|
||
## Uninstall on Debian or Ubuntu | ||
|
||
To uninstall Pelican, run the following command | ||
|
||
```bash | ||
sudo apt-get remove pelican | ||
``` | ||
|
||
## Verify Pelican is installed | ||
|
||
Refer to the [previous section](./rhel.mdx#verify-pelican-is-installed) on how to verify that Pelican is successfully installed. | ||
|
||
## Next steps | ||
|
||
If you want to use Pelican client to download or upload files, refer to [Client Usage](../client-usage.mdx) to get started. If you want to serve a Pelican server, refer to [Serve a Pelican Origin](../serving_an_origin.mdx) and [Serve a Pelican Federation](../serving_a_federation.mdx). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
# Run Pelican Server using Docker image | ||
|
||
This document explains how to run a Pelican server using Pelican docker image. If you are installing Pelican to use its client functionalities (download or upload a file), refer to previous sections to download and install a Pelican binary instead. | ||
|
||
## Before starting | ||
|
||
Pelican builds separate images for each Pelican server components, e.g. origin, cache, director, and registry. Depending on which Pelican server you want to run, you need to select a different docker image from the list below. | ||
|
||
* Pelican origin server: `hub.opensciencegrid.org/pelican_platform/origin:latest` | ||
* Pelican cache server: `hub.opensciencegrid.org/pelican_platform/cache:latest` | ||
* Pelican director server: `hub.opensciencegrid.org/pelican_platform/director:latest` | ||
* Pelican registry server: `hub.opensciencegrid.org/pelican_platform/registry:latest` | ||
|
||
## Run Pelican server via Docker CLI | ||
|
||
This section shows you how to run various Pelican server images using the Docker CLI. If you haven't installed Docker engine, follow the [documentation from Docker](https://docs.docker.com/get-docker/) to install the Docker engine. | ||
|
||
|
||
### Run Pelican origin server | ||
|
||
To run the latest pelican origin server, run the following command: | ||
|
||
```bash | ||
docker run -it -p 8444:8444 -p 8443:8443 -v /path/to/your/data/:/tmp/pelican --name=pelican-origin hub.opensciencegrid.org/pelican_platform/origin:latest serve -v /tmp/pelican:/foo/bar | ||
``` | ||
|
||
Where: | ||
|
||
* `docker run` is a Docker CLI command that runs a new container from an image | ||
* `-it` (`--interactive --tty`) runs the container in interactive mode and uses a tty terminal | ||
* `-p <host-port>:<container-port>` (`--publish`) publishes a container’s port(s) to the host, allowing you to reach the container’s port via a host port. In this case, we can reach the container’s port `8444` via the host’s port `8444`. Note that the web engine of the Pelican server runs on port `8444` by default, and the file transfer endpoint of the Pelican server runs on port `8443` by default. | ||
* `-v <host-location>:<container-location>` (`--volume`) binds mount a volume from the host location(s) to the containter's location(s). This allow you to share files in your host machine to the container. In this case, we bind `/path/to/your/data/` on your host machine to `/tmp/pelican` in the container. You need to replace `/path/to/your/data/` to the directory where your data to publish is located. | ||
* `--name` assigns a logical name to the container (e.g. pelican-origin). This allows you to refer to the container by name instead of by ID. | ||
* `hub.opensciencegrid.org/pelican_platform/origin:latest` is the image to run | ||
* `serve` is the command to pass to pelican binary. You can also pass arguments to Pelican by appending the argument after `serve` command. Equivalent of running `pelican origin serve` | ||
* `-v /tmp/pelican:/foo/bar` is the Pelican argument to bind `/tmp/pelican` directory in the container as namespace `/foo/bar` in Pelican. You need to change `/foo/bar` to a meaningful path that can represent your data, e.g. `/chtc/public-data`. | ||
|
||
|
||
### Run Pelican cache server | ||
|
||
To run the latest pelican cache server, run the following command: | ||
|
||
```bash | ||
docker run -it -p 8444:8444 -p 8442:8442 --name=pelican-cache hub.opensciencegrid.org/pelican_platform/cache:latest serve | ||
``` | ||
|
||
> Note that Pelican docker image currently does not support binding a directory on your host machine as the directory for Pelican cache to store cache files, i.e. using `-v /host/machine:/run/pelican/cache/location`. We will have an update once we officialy support this flow. | ||
|
||
### Run Pelican director server | ||
|
||
To run the latest pelican director server, run the following command: | ||
|
||
```bash | ||
docker run -it -p 8444:8444 --name=pelican-director hub.opensciencegrid.org/pelican_platform/director:latest serve | ||
``` | ||
|
||
> Note that to successfully run a Pelican director server, additional configuration is required. Follow [Serve a Pelican Federation](../serving_a_federation.mdx) for instructions. | ||
|
||
### Run Pelican registry server | ||
|
||
To run the latest pelican registry server, run the following command: | ||
|
||
```bash | ||
docker run -it -p 8444:8444 --name=pelican-registry hub.opensciencegrid.org/pelican_platform/registry:latest serve | ||
``` | ||
|
||
> Note that to successfully run a Pelican registry server, additional configuration is required. Follow [Serve a Pelican Federation](../serving_a_federation.mdx) for instructions. | ||
|
||
|
||
## Stop Pelican container | ||
|
||
To stop the Pelican container, run the following command: | ||
|
||
```bash | ||
# The `docker ps` command shows the processes running in Docker | ||
docker ps | ||
|
||
# This will display a list of containers that looks like the following: | ||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES | ||
0be1a304b5d7 hub.opensciencegrid.org/pelican_platform/director:latest "/bin/sh" 1 hour ago Up 1 hour 0.0.0.0:8444->8444/tcp pelican-director | ||
|
||
# To stop the pelican container run the command | ||
# docker stop <container-ID> or use | ||
# docker stop <container-name>, which is `pelican-director` as previously defined | ||
docker stop pelican-director | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
# Install Pelican on macOS | ||
|
||
This document explains how to install Pelican on macOS. | ||
|
||
### Install Pelican as a standalone binary | ||
|
||
1. Navigate to [Pelican download page](../install.mdx) and select the Pelican macOS binary you want to install. | ||
|
||
2. In **Operating System** section, select **macOS**. In **Architechtures** section, select **X84_64** (Intel) or **ARM64** (ARM) depending on the architecture of your machine. | ||
|
||
3. In the list of candidates, copy the link to `pelican_Darwin_arm64.tar.gz` if you select **ARM64** (ARM), or `pelican_Darwin_x86_64.tar.gz` if you select **X84_64** (Intel). | ||
|
||
4. Change the following command with the link to the binary you copied in the previous step and run the command | ||
|
||
```bash | ||
wget <replace-with-the-link-you-copied> | ||
tar -zxvf pelican_Darwin_arm64.tar.gz // ARM user | ||
``` | ||
|
||
Note that you need to replace `pelican_Darwin_arm64.tar.gz` with `pelican_Darwin_x86_64.tar.gz` if you are running an Intel machine. | ||
|
||
Example to install Pelican binary for an Intel machine: | ||
|
||
```bash | ||
curl -O https://github.com/PelicanPlatform/pelican/releases/download/v7.5.8/pelican_Darwin_x86_64.tar.gz | ||
tar -zxvf pelican_Darwin_x86_64.tar.gz | ||
``` | ||
|
||
5. At this step, you don't actually install Pelican, you just extract the binary. It is recommended that you add Pelican binary to your `PATH` environment variable to allow Pelican to be called from your command line. | ||
|
||
## Verify Pelican is installed | ||
|
||
Refer to the [previous section](./rhel.mdx#verify-pelican-is-installed) on how to verify that Pelican is successfully installed. | ||
|
||
## Next steps | ||
|
||
If you want to use Pelican client to download or upload files, refer to [Client Usage](../client-usage.mdx) to get started. If you want to serve a Pelican server, refer to [Serve a Pelican Origin](../serving_an_origin.mdx) and [Serve a Pelican Federation](../serving_a_federation.mdx). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd be careful in making this claim. While we can technically run the pelican binary on Windows, most of the functionality is turned of or compiled out. It's not as bad with MacOS, but we're still largely untested there as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you think is is for windows only or the claim that we support the OS? If it's former, then I can add a note to Windows item suggesting that we only support client functionality. This would be the same thing for macOS where I don't think xrootd works well with mac and people should be required to build xrootd from source
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I haven't even tested the client on Windows and I've not examined the extent of our windows tests. Does the client fully work there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've bugged @joereuss12 to run the
.exe
program we generated and see if that works