Installing a Virtual Tape Library: Difference between revisions

From Proxmox Backup Server
Jump to navigation Jump to search
No edit summary
(Add a VTL/Debian update section)
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==


Proxmox Backup Server 1.1 and later include support for tape backup, allowing you to save Datastores to tape, and manage tape libraries and tape pools.
Proxmox Backup Server includes support for tape backup, allowing you to save 'Datastores' to tape, and manage tape libraries and tape pools.


Since testing backup and restore with real tape hardware takes a lot of time, it can be a good idea to use a faster Virtual Tape Library (VTL) for evaluation and for testing
Since testing backup and restore with real tape hardware takes a lot of time, it can be a good idea to use a faster Virtual Tape Library (VTL) for evaluation and for testing
of the correct allocation and retention policies for your use case.
of the correct allocation and retention policies matching your use case.


One example of such a VTL is the [https://www.quadstor.com/virtual-tape-library.html QuadStor VTL].
One example of such a VTL is the [https://www.quadstor.com/virtual-tape-library.html QUADStor VTL].
This tutorial will guide you through the installation and configuration of a QuadStor VTL, and also show you how to integrate it into Proxmox Backup Server.
This tutorial will guide you through the installation and configuration of a QUADStor VTL, and also show you how to integrate it into Proxmox Backup Server.


== Installing the QuadStor VTL ==
== Installing the QUADStor VTL ==


It's recommended to install it on a separate Virtual Machine (for example, on a Proxmox VE installation).
This tutorial assumes the use of an offical [https://www.debian.org/ Debian] 12 (Bookworm), for example, running as a virtual machine on a Proxmox VE host.


This tutorial assumes the use of a VM with Debian 11 (Bullseye) as the OS and '10.20.1.59' as its IP address. Two disks are needed, one for the OS, one (larger) for the VTL as storage.
Two disks are needed. One for the Debian OS installation (for example 16 GiB), and a larger one to store virtual tapes on it. A disk of at least 220 GiB allows you to fully utilize the capacity of two LTO-1 (100 GB) tapes. While real LTO-1 is not [https://pbs.proxmox.com/docs/tape-backup.html#supported-hardware supported hardware] by Proxmox Backup Server, it is still convenient for testing purposes due to its low capacity, compared to modern types like LTO-8 with 12 TB.


The installation instructions are based on the [https://www.quadstor.com/vtlsupport/145-installation-on-rhel-centos-sles-debian.html official QuadStor documentation] for Debian 9, but they will also work for Debian 11.
For our example, Debian has the static IP address '10.20.1.59'.
 
The following instructions are performed in the Debian VM and are based on the [https://www.quadstor.com/vtlsupport/145-installation-on-rhel-centos-sles-debian.html official QUADStor documentation].


# Install supporting packages:
# Install supporting packages:
#:<pre>
#:<pre>
#::apt update
#::apt update
#::apt install uuid-runtime build-essential sg3-utils apache2 psmisc linux-headers-`uname -r`
#::apt install uuid-runtime build-essential sg3-utils apache2 gzip xz-utils postgresql libpq-dev psmisc linux-headers-`uname -r`
#::a2enmod cgi
#::a2enmod cgi
#::systemctl restart apache2</pre>
#::systemctl restart apache2</pre>
# Now you can download and install the VTL packages:
# Look up the current Debian package version number on the [https://www.quadstor.com/vtl-extended-edition-downloads.html QUADStor website] and set it as a variable. For example:
#:<pre>
#:<pre>
#::wget -c https://www.quadstor.com/vtldownloads/quadstor-vtl-ext-3.0.76.1-debian-x86_64.deb
#::version=3.0.79.15</pre>
#::apt install ./quadstor-vtl-ext-3.0.76.1-debian-x86_64.deb</pre>
# Download and install the package, this may take a while:
#: Installation can take a while.
#:<pre>
# Following this, the daemon should be started/enabled automatically. If not, you can enable and start it manually with: <code>systemctl enable --now quadstorvtl.service</code>
#::wget -c https://www.quadstor.com/vtldownloads/quadstor-vtl-ext-$version-debian12-x86_64.deb
# Finally, the VM needs to be rebooted: <code>reboot</code>
#::apt install ./quadstor-vtl-ext-$version-debian12-x86_64.deb</pre>
# Run <code>systemctl enable --now quadstorvtl.service</code> and wait for its completion to verify that no errors occur.
# Finally, reboot the VM: <code>reboot</code>
 
You can now access the VTL configuration interface over HTTP using your web browser. In our example, we visit http://10.20.1.59.
 
== Updating the QUADStor VTL and Debian ==
 
If the VTL is used for a longer period of time, we recommend installing available updates on a regular basis.
 
''' Updating Debian '''
: For example, run <code>apt update</code>, <code>apt full-upgrade</code> and <code>reboot</code>. If your Debian major version will not receive any more updates in the future, we recommend setting up a new VM and VTL for the sake of simplicity.


You can now access the VTL configuration interface over HTTP using your browser. In our example, we'd use http://10.20.1.59.
''' Updating QUADStor VTL '''
: Run <code>apt remove quadstor-vtl-ext</code> and follow the [[#Installing_the_QUADStor_VTL|installation section]] above, starting with step 2.


== Configuring the VTL ==
== Configuring the VTL ==


These steps are described in more detail in the [https://www.quadstor.com/vtl-documentation.html official QuadStore documentation], but the minimum required steps are repeated here.  
The following configurations are performed in the QUADStor web interface and are the minimum required ones. Details can be found in the [https://www.quadstor.com/vtl-documentation.html official QUADStor documentation].
 
:1. Go to the tab 'Physical Storage'
::The table should show the empty disk, click on 'Add' and submit the 'Default' 'Storage Pool'.
 
:2. Go to 'Device Definitions'
::You need a 'Changer' and 'Drive' definition. Their exact properties should not be that important for our virtualization purpose. If in doubt, you can also download and import the templates [https://www.quadstor.com/uscripts/changer-definitions.txt changer-definitions.txt] and [https://www.quadstor.com/uscripts/drive-definitions.txt drive-definitions.txt] from https://www.quadstor.com/uscripts/.
 
:3. Go to 'Virtual Libraries'
::Add a Virtual Tape Library (VTL) by setting your desired properties. For example:
{| class="wikitable" style="margin-left: 80px"
! Field !! Value !! Note
|-
| VTL Name || style="text-align:right" | my-tape-lib ||
|-
| Changer Definition || style="text-align:right" | IBM_3584 ||
|-
| Drive Definition || style="text-align:right" | IBM_LTO1 || for example LTO1 to virtualize a capacity of 100 GB
|-
| Number of VDrives || style="text-align:right" | 2 || concurrently usable tapes
|-
| Number of VSlots || style="text-align:right" | 20 || to park tapes
|-
| Number of IE Ports || style="text-align:right" | 4 || for tape import/export
|}
 
:4. Finally, click 'Add VCartridge'
::Specify how many tapes should be inserted into the VSlots now (for example, 2). The 'Label/Prefix' (for example, tape00) must have 6 characters, including a zero for the auto-numbering scheme.


# The first thing to configure is the 'Physical Storage'.
The newly created VTL can now be added to a Proxmox Backup Server instance.
#: For this, you need to add a new disk to the VM.
# After that, you can click on 'Physical Storage' and 'Rescan'.
#: Your newly added, empty disks should show up with their Vendor, Model and Serial information.
#: Click on 'Add'.
# Now select a 'Storage Pool' (there should be one called 'Default') and click 'Submit'.
#: This will initialize the disk so that it can be used as a storage target for the virtual tapes.
# Now you can create a Virtual Tape Library (VTL). Click on 'Virtual Libraries' and 'Add VTL'.
#: Give it a name under 'VTL Name' and select:
#:* The Vendor/Model you want to simulate (best if this is as close as possible to your real tape hardware for testing).
#:* The type and number of VDrives (Virtual Drives). These define the actual tape drives inside the library.
#:* The number of VSlots (Virtual Slots), which is the number of slots that can hold tapes.
#:* The number of 'IE Ports' (Import/Export slots).
#::The remaining options are not so important and can be left on their default.
#: For this example, let's choose the name 'MYTAPELIB' and the library 'IBM 3584 Ultra Scalable Library' with 2 VDrives 'IBM 3580 Ultrium 8'.
#* Click on 'Submit'.
# Now you need to add some virtual tapes by clicking 'Add VCartridge'.
#* Choose the number you want (this will matter for the auto-numbering scheme) and a Prefix, for example 'TAPES0', and then click on 'Submit'.
# After this, you can add the newly created VTL to a Proxmox Backup Server instance.


== Adding the VTL to Proxmox Backup Server ==
== Adding the VTL to Proxmox Backup Server ==


The following commands must all be executed on the Proxmox Backup Server instance.
The following commands must all be executed on the Proxmox Backup Server.


=== Making devices available over iSCSI ===
=== Making devices available over iSCSI ===


# The VTL will be added to Proxmox Backup Server via iSCSI, so you first make sure that the open-iscsi package is installed:
# Since the VTL will be added to Proxmox Backup Server via iSCSI, make sure that the open-iscsi package is installed:
#:<pre>apt install open-iscsi</pre>
#:<pre>apt install open-iscsi</pre>
# To automatically log in to the iSCSI targets on boot, you have to carry out the following steps:
# To automatically log in to the iSCSI targets on boot, carry out the following steps:
## modify the iSCSI config in <code>/etc/iscsi/iscsid.conf</code> and change <code>node.startup = manual</code> to <code>node.startup = automatic</code>
## open the iSCSI config (<code>editor /etc/iscsi/iscsid.conf</code>) and change <code>node.startup = manual</code> to <code>node.startup = automatic</code>
## then restart <code>iscsid</code> by executing <code>systemctl restart open-iscsi.service</code>
## then restart <code>iscsid</code> by executing <code>systemctl restart open-iscsi.service</code>
# Now you need to discover the VTL tape drive and changer devices with <code>iscsiadm</code>. Please do not forget to adapt the IP address, if you chose a different one for the Quadstore VTL VM.
# Now discover the changer and drive iSCSI targets via <code>iscsiadm</code>. Please do not forget to adapt the IP address if you chose a different one for your QUADStor VTL VM.
#:<pre>iscsiadm -m discovery -t st -p 10.20.1.59</pre>
#:<pre>iscsiadm -m discovery -t st -p 10.20.1.59</pre>
#:The resulting output should look like this:
#:The resulting output should look like this:
#::<pre>
#::<pre>
#:::10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.MYTAPELIB.autoloader
#:::10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.my-tape-lib.autoloader
#:::10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.MYTAPELIB.drive2
#:::10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive1
#:::10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.MYTAPELIB.drive1</pre>
#:::10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive2</pre>
# You have to log in to each of these devices with
# To populate <code>/dev/tape/</code> with all discovered devices, run <code>iscsiadm -m discovery -l -p 10.20.1.59</code>. Alternatively, you can also log in manually:
#:<pre>
#:<pre>
#::iscsiadm -m node --targetname iqn.2006-06.com.quadstor.vtl.MYTAPELIB.autoloader --login
#::iscsiadm -m node -l --targetname iqn.2006-06.com.quadstor.vtl.my-tape-lib.autoloader
#::iscsiadm -m node --targetname iqn.2006-06.com.quadstor.vtl.MYTAPELIB.drive1 --login
#::iscsiadm -m node -l --targetname iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive1
#::iscsiadm -m node --targetname iqn.2006-06.com.quadstor.vtl.MYTAPELIB.drive2 --login</pre>
#::iscsiadm -m node -l --targetname iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive2</pre>
#: This will populate <code>/dev</code> with the correct devices.


=== Adding the Changer and Drives to Proxmox Backup Server ===
=== Adding the Changer and Drives to Proxmox Backup Server ===


You can now add the devices from the Proxmox Backup Server WebUI:
In the web interface of your Proxmox Backup Server:
# Go to 'Tape Backup' -> 'Changers'
# Go to 'Tape Backup' -> 'Changers'
# Click on 'Add', give the changer a name, and select the path (should be autofilled).
#: Click on 'Add', give the changer a name (for example, autoloader), and select its path in the drop-down list.
# Now go to 'Tape Backup' -> 'Drives'
# Go to 'Tape Backup' -> 'Drives'
#* For each drive, click on 'Add' and choose a name, the corresponding changer, the index of the drive inside the changer (Drive Number), and the correct path.
## Click on 'Add', give the drive a name (for example, drive1), select the changer and keep the 'Drive Number' '0'. The correct 'Path' can be chosen by comparing the serial numbers with those in the QUADStor web interface, under 'Virtual Libraries'.
#* Verify the index by comparing the serial numbers in the Proxmox Backup Server Web UI and the VTL Web UI.
## When adding a second drive, proceed with 'Drive Number' '1'.


After adding the drives and changers, you can use the VTL like a normal tape library in Proxmox Backup Server.
The VTL can now be used like a normal library in Proxmox Backup Server.


[[Category:HOW-TO]]
[[Category:HOW-TO]]

Latest revision as of 13:38, 13 August 2024

Introduction

Proxmox Backup Server includes support for tape backup, allowing you to save 'Datastores' to tape, and manage tape libraries and tape pools.

Since testing backup and restore with real tape hardware takes a lot of time, it can be a good idea to use a faster Virtual Tape Library (VTL) for evaluation and for testing of the correct allocation and retention policies matching your use case.

One example of such a VTL is the QUADStor VTL. This tutorial will guide you through the installation and configuration of a QUADStor VTL, and also show you how to integrate it into Proxmox Backup Server.

Installing the QUADStor VTL

This tutorial assumes the use of an offical Debian 12 (Bookworm), for example, running as a virtual machine on a Proxmox VE host.

Two disks are needed. One for the Debian OS installation (for example 16 GiB), and a larger one to store virtual tapes on it. A disk of at least 220 GiB allows you to fully utilize the capacity of two LTO-1 (100 GB) tapes. While real LTO-1 is not supported hardware by Proxmox Backup Server, it is still convenient for testing purposes due to its low capacity, compared to modern types like LTO-8 with 12 TB.

For our example, Debian has the static IP address '10.20.1.59'.

The following instructions are performed in the Debian VM and are based on the official QUADStor documentation.

  1. Install supporting packages: 
    apt update
    

    apt install uuid-runtime build-essential sg3-utils apache2 gzip xz-utils postgresql libpq-dev psmisc linux-headers-`uname -r`
    

    a2enmod cgi
    

    systemctl restart apache2
  2. Look up the current Debian package version number on the QUADStor website and set it as a variable. For example: 
    version=3.0.79.15
  3. Download and install the package, this may take a while: 
    wget -c https://www.quadstor.com/vtldownloads/quadstor-vtl-ext-$version-debian12-x86_64.deb
    

    apt install ./quadstor-vtl-ext-$version-debian12-x86_64.deb
  4. Run systemctl enable --now quadstorvtl.service and wait for its completion to verify that no errors occur.
  5. Finally, reboot the VM: reboot

You can now access the VTL configuration interface over HTTP using your web browser. In our example, we visit http://10.20.1.59.

Updating the QUADStor VTL and Debian

If the VTL is used for a longer period of time, we recommend installing available updates on a regular basis.

Updating Debian

For example, run apt update, apt full-upgrade and reboot. If your Debian major version will not receive any more updates in the future, we recommend setting up a new VM and VTL for the sake of simplicity.

Updating QUADStor VTL

Run apt remove quadstor-vtl-ext and follow the installation section above, starting with step 2.

Configuring the VTL

The following configurations are performed in the QUADStor web interface and are the minimum required ones. Details can be found in the official QUADStor documentation.

1. Go to the tab 'Physical Storage'
The table should show the empty disk, click on 'Add' and submit the 'Default' 'Storage Pool'.
2. Go to 'Device Definitions'
You need a 'Changer' and 'Drive' definition. Their exact properties should not be that important for our virtualization purpose. If in doubt, you can also download and import the templates changer-definitions.txt and drive-definitions.txt from https://www.quadstor.com/uscripts/.
3. Go to 'Virtual Libraries'
Add a Virtual Tape Library (VTL) by setting your desired properties. For example:
Field Value Note
VTL Name my-tape-lib
Changer Definition IBM_3584
Drive Definition IBM_LTO1 for example LTO1 to virtualize a capacity of 100 GB
Number of VDrives 2 concurrently usable tapes
Number of VSlots 20 to park tapes
Number of IE Ports 4 for tape import/export
4. Finally, click 'Add VCartridge'
Specify how many tapes should be inserted into the VSlots now (for example, 2). The 'Label/Prefix' (for example, tape00) must have 6 characters, including a zero for the auto-numbering scheme.

The newly created VTL can now be added to a Proxmox Backup Server instance.

Adding the VTL to Proxmox Backup Server

The following commands must all be executed on the Proxmox Backup Server.

Making devices available over iSCSI

  1. Since the VTL will be added to Proxmox Backup Server via iSCSI, make sure that the open-iscsi package is installed: 
    apt install open-iscsi
  2. To automatically log in to the iSCSI targets on boot, carry out the following steps:
    1. open the iSCSI config (editor /etc/iscsi/iscsid.conf) and change node.startup = manual to node.startup = automatic
    2. then restart iscsid by executing systemctl restart open-iscsi.service
  3. Now discover the changer and drive iSCSI targets via iscsiadm. Please do not forget to adapt the IP address if you chose a different one for your QUADStor VTL VM. 
    iscsiadm -m discovery -t st -p 10.20.1.59
    The resulting output should look like this: 
    10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.my-tape-lib.autoloader
    

    10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive1
    

    10.20.1.59:3260,1 iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive2
  4. To populate /dev/tape/ with all discovered devices, run iscsiadm -m discovery -l -p 10.20.1.59. Alternatively, you can also log in manually: 
    iscsiadm -m node -l --targetname iqn.2006-06.com.quadstor.vtl.my-tape-lib.autoloader
    

    iscsiadm -m node -l --targetname iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive1
    

    iscsiadm -m node -l --targetname iqn.2006-06.com.quadstor.vtl.my-tape-lib.drive2

Adding the Changer and Drives to Proxmox Backup Server

In the web interface of your Proxmox Backup Server:

  1. Go to 'Tape Backup' -> 'Changers'
    Click on 'Add', give the changer a name (for example, autoloader), and select its path in the drop-down list.
  2. Go to 'Tape Backup' -> 'Drives'
    1. Click on 'Add', give the drive a name (for example, drive1), select the changer and keep the 'Drive Number' '0'. The correct 'Path' can be chosen by comparing the serial numbers with those in the QUADStor web interface, under 'Virtual Libraries'.
    2. When adding a second drive, proceed with 'Drive Number' '1'.

The VTL can now be used like a normal library in Proxmox Backup Server.

Retrieved from "https://pbs.proxmox.com/wiki/index.php?title=Installing_a_Virtual_Tape_Library&oldid=133"