Skip to content

Latest commit

 

History

History
188 lines (151 loc) · 5.48 KB

README.md

File metadata and controls

188 lines (151 loc) · 5.48 KB
Raw

incus-auto

incus-auto is automating deployment system using Incus.

Requirements

  • Incus (or LXD)
  • (Option) Nested Virtualization
    • If using VM on VM (Virtual machine)
    • Example of Hyper-V:
      • Set-VMProcessor -VMName VM-NAME -ExposeVirtualizationExtensions $true

Features

  • Automatic source=agent:config for Incus VM

Setup

  • Install Incus

  • Edit /etc/subuid and /etc/subgid for incus if needed

    root:1000:65535
root:1000000:65536

  • Initialize Incus

    • Example: incus admin init
      • Default storage pool default is used.
      • Default network bridge incusbr0 is not used.

  • Install pigz for performance

    • apt-get install pigz

  • Run incus config set images.compression_algorithm pigz

  • Install python3-docopt, python3-schema, python3-yaml

    • Debian series (Ubuntu, etc.):
      • apt-get install python3 python3-docopt python3-schema python3-yaml
    • Fedora series (RHEL, RockyLinux, AlmaLinux, etc.):
      • yum install epel-release
      • yum install python3 python3-docopt python3-schema python3-pyyaml

  • Install incus-auto

    • Example:
      • install -m 755 bin/incus-auto /usr/local/bin/

  • Create storage pool when using other than "default" pool

  • Prepare remote server if needed

  • Create incus-auto.yaml and incus-auto.override.yaml

    • Define config (profile, network, disk, ...)
      • Define config
        • (Option) Set default_pool
      • (Option) Define buildimage
      • Define host
    • Example: ./example/gfarm/incus-auto.*yaml
      • Detail: ./example/gfarm/README-gfarm.md

  • (Recommendation) Install apt-cacher-ng and set http_proxy to accelerate

  • Initialize profile and network

    • incus-auto init

  • Build images

    • incus-auto build -a

  • Launch containers or virtual machines

    • incus-auto launch -a

Add remote Incus server

  • client certificate
    • Incus
      • ~/.config/incus/client.crt
    • LXD
      • ~/snap/lxd/common/config/client.crt
  • @server (for example)
    • Incus
      • incus config set core.https_address "[::]:8443"
      • incus config trust add-certificate client1-client.crt --name client1
    • LXD
      • lxc config set core.https_address "[::]:18443"
      • lxc config trust add client2-client.crt --name client2
  • @client (for example)

Using http_proxy (apt-cacher-ng, etc.)

Create incus-auto.override.yaml

config:
  envvar:
    http_proxy: http://<apt-cacher-ng IP address>:3142
    https_proxy: http://<apt-cacher-ng IP address>:3142

Commands

See:

incus-auto -h

Incus mirror servers

Firewall

WSL2

  • When using VM, Custom WSL2-Linux-Kernel is required

    • "vhost virtio-vsock driver" is required
      • Error: Failed to run: modprobe -b vhost_vsock: exit status 1 (modprobe: FATAL: Module vhost_vsock not found in directory /lib/modules/5.15.146.1-microsoft-standard-WSL2)
    • Windows 11 is required
      • for nestedVirtualization=true (default) (.wslconfig)
      • not work on Windows 10

  • How to Build custom WSL2-Linux-Kernel (to enable vhost_vsock module)

sudo apt update
sudo apt install git bc build-essential flex bison libssl-dev libelf-dev dwarves
uname -a
Linux DVL3-01 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

# not set .1
KVER=5.15.133
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-${KVER}.tar.xz
tar xf linux-${KVER}.tar.xz

pushd linux-${KVER}
wget -O arch/x86/configs/config-wsl \
https://raw.githubusercontent.com/microsoft/WSL2-Linux-Kernel/linux-msft-wsl-5.15.y/arch/x86/configs/config-wsl

# Create a duplicate to modify
cp arch/x86/configs/config-wsl arch/x86/configs/config-wsl-custom

sudo apt install libncurses-dev
make KCONFIG_CONFIG=arch/x86/configs/config-wsl-custom menuconfig
  • Device Drivers -> VHOST drivers -> vhost virtio-vsock driver
make KCONFIG_CONFIG=arch/x86/configs/config-wsl-custom -j$(nproc)
sudo make KCONFIG_CONFIG=arch/x86/configs/config-wsl-custom modules_install

mkdir -p /mnt/c/Users/USERNAME/wsl2-kernel
cp arch/x86/boot/bzImage /mnt/c/Users/USERNAME/wsl2-kernel/bzImage-5.15.133-WSL2-custom

echo vhost_vsock | sudo tee -a /etc/modules

vi  /mnt/c/Users/USERNAME/.wslconfig
[wsl2]
kernel=C:\\\\Users\\\\USERNAME\\\\wsl2-kernel\\\\bzImage-5.15.133-WSL2-custom

  • Start Ubuntu on WSL2

  • in Ubuntu

    • sudo iptables -P FORWARD ACCEPT
    • Edit /etc/wsl.conf
    [boot]
memory=16GB
systemd=true

  • wsl --shutdown at Windows Powershell

  • Start Ubuntu on WSL2

  • Install Incus