Allow callers to opt-out of 'qemu-img' autodetecting the format.
Currently not supported to be done together with untrusted, because it
can lead to less checks being done. Could be further refined (e.g.
disallow only untrusted together with format being 'raw') should the
need arise.
For 'subvol' format, the checking is handled outside of 'qemu-img' of
course, based on whether it is a directory or not.
Currently, there is a fallback to 'raw' should the format not be among
the ones allowed for the 'pve-qm-image-format' standard option. This
is to reduce potential for fallout, in particular for the plan to
change the base plugin's volume_size_info() to pass in the expected
format when calling file_size_info() too.
While not explicitly part of the storage plugin API, the 'untrusted'
parameter is now in a different place, so a compat check is added for
external plugins that might've still used it.
Breaks for qemu-server needed (if we don't want to just rely on the
compat check).
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Add a SAFE_CHAR_WITH_WHITESPACE_CLASS_RE sister variant of the shared
SAFE_CHAR_CLASS_RE shared regex to the base storage module, this use
case exist is a generic one after all, and use swap the untaint method
that parses the file a disk references to it.
Note that this is only the disk file name from inside the archive and
thus during the extraction to a staging/working directory, from there
it will be imported as volume allocated by the common storage system,
and thus follow our ordinary volume name scheme.
Improves disk detection when importing, e.g., the from upstream
provided GNS3 OVA.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
introducing a separate regex that only contains ova, since
upload/downloading ovfs does not make sense (since the disks are then
missing).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
since we want to handle ova files (which are only ovf+images bundled in
a tar file) for import, add code that handles that.
we introduce a valid volname for files contained in ovas like this:
storage:import/archive.ova/disk-1.vmdk
by basically treating the last part of the path as the name for the
contained disk we want.
in that case we return 'import' as type with 'vmdk/qcow2/raw' as format
(we cannot use something like 'ova+vmdk' without extending the 'format'
parsing to that for all storages/formats. This is because it runs
though a verify format check at least once)
we then provide a function to use for that:
* extract_disk_from_import_file: this actually extracts the file from
the archive. Currently only ova is supported, so the extraction with
'tar' is hardcoded, but again we can easily extend/modify that should
we need to.
we currently extract into the either the import storage or a given
target storage in the images directory so if the cleanup does not
happen, the user can still see and interact with the image via
api/cli/gui
we have to modify the `parse_ovf` a bit to handle the missing disk
images, and we parse the size out of the ovf part (since this is
informal only, it should be no problem if we cannot parse it sometimes)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
in DirPlugin and not Plugin (because of cyclic dependency of
Plugin -> OVF -> Storage -> Plugin otherwise)
only ovf is currently supported (though ova will be shown in import
listing), expects the files to not be in a subdir, and adjacent to the
ovf file.
listed will be all ovf/qcow2/raw/vmdk files.
ovf because it can be imported, and the rest because they can be used
in the 'import-from' part of qemu-server.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
by letting it run through 'file_size_info' as 'untrusted', since that
does the necessary checks. We do this so we don't accidentally
up/download a file that is not a valid iso
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
this allows checking some extra attributes for images which come from
a potentially malicious source.
since file_size_info is not part of the plugin API, no API bump is
needed. if desired, a similar check could also be implemented in
volume_size_info, which would entail bumping both APIVER and APIAGE
(since the additional parameter would make checking untrusted volumes
opt-in for external plugins).
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
A popular ISO compressed exclusively with bz2 is OPNsense [2].
Since this requires adding `bz2` to the list of known compression
formats we add decompression methods for vmz and tar.
[2] https://opnsense.org/download/
Suggested-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Tested-By: Aaron Lauterer <a.lauterer@proxmox.com>
The storage API version has been bumped to at least 9 since
libpve-storage = 7.0-4. If the source node is on Proxmox VE 8, where
this change will come in, then the target node can be assumed to be
running either Proxmox VE 8 or, during upgrade, the latest version of
Proxmox VE 7.4, so it's safe to assume a storage API version of at
least 9 in all cases.
As reported by Maximiliano, the fact that the 'apiinfo' call was
guarded with a quiet eval could lead to strange errors for replication
on a customer system where an SSH connection could not always be
established, because the target's API version would fall back to 1.
Because of that, the '-base' argument would be missing for the import
call on the target which would in turn lead to an error about the
target ZFS volume already existing (rather than doing an incremental
sync).
Reported-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
both STDOUT and STDERR are written into `$info` which is then parsed for
IP and port of the target socket listening.
when the ports file can't be locked immediately `trying to acquire
lock...` is printed on STDERR and in turn written into `$info`.
trying to parse the IP then fails, resulting in a migration or
replication failing.
the bare open3 call is replaced by the run_command wrapper from
pve-common to use a safe wrapper around open3 with the same
functionality.
STDERR is read separatey from STDOUT and the last line of STDERR is
kept in case of errors.
Fixes: 57acd6a ("fix #1452: also log stderr of remote command with
insecure storage migration")
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
The get_guest_import fn is currently a private/internal API and should
not yet be exposed as a public part of the storage API.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Add a new 'import' content type which will be the corner stone for a
better API and UI integrated way to import virtual guests into Proxmox
VE.
For starters this will be used to implement a ESXi adapter, so that
those VMs can get imported nicely.
Later we want to integrate the OVF/OVA import skeletons we got in
qemu-server to something more usable here.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
[ TL: add more commit message with some background ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With this patch Proxmox now tries to login to all discovered portals
in case some of them are not logged yet.
In case of multipath configuration when initially configured portal is
missing for some reason Proxmox don't lose iSCSI storage now and can
successfully restore iSCSI connection between reboots.
Signed-off-by: Yuri Konotopov <ykonotopov@gnome.org>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
With that the if/else branches are pretty clear, so drop the rather
noisy comments that describe what one can get from the code already
anyway.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
dd supports a 'status' flag, which enables it to show the copied bytes,
duration, and the transfer rate, which then get printed to stderr.
Signed-off-by: Leo Nunner <l.nunner@proxmox.com>
adds information for how to decompress isos.
generates the compressor regex from a list of comression formats (to
avoid redundancy)
extends the download_url wtih the functionality to handley compression
for images
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
removed Data::Dumper and a newline
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
re-added the newline, but fixed up the indentation at that line..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Other places like replication explicitly state whether byte or bit
are used as a unit rather than only the abbreviation.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
