Currently, the helper would not distinguish between different kinds
of errors. Instead of relying on an error, list the images and check
there.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
For now, only 'raw+size' is supported and it's not possible to
export/import with snapshots. The volume or snapshot is exported or
imported via the corresponding 'rbd' commands.
Introducing an 'rbd' transport format might be feasible for more
complete (i.e. with snapshots, incremental) transfer between two RBD
storages.
Use the '--dest-pool' switch rather than '-p' for import, because the
latter is deprecated.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Current export implementations luckily seems to not run into this
issue yet. However, for the upcoming implementation for RBD, mapping a
volume would print the device path to STDOUT, thus messing up the
export stream.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
The plugin API does not require call context detection for the
returned value of the path() method. See other plugins like
ISCSIDirect/ZFS that do not implement it. So do not expect it for
external plugins either.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
in case this gets called with an explicit format that is none of:
- 'auto-detect'
- 'subvol'
- a member of the list of known "qemu" formats
this should only affect third-party storage plugins that either call this
directly with a format, or via inherited code that gets a format from
parse_volname and passes it to file_size_info.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
in case of an array context, it should also return the format, else a caller
might assume it failed.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
since `volume_size_info` passes the parsed format to `file_size_info`, which
prints a warning if the format is undef before falling back to auto-detection,
and these should always be treated as raw files anyway.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
otherwise, `volname.vmdk.raw` would be picked up as a `vmdk` formatted
image, which may not be desired.
Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
The only (transitive) caller that seems to be interested in the format
is the API endpoint for content listing.
The warning about not being able to query in the expected format might
not be seen by consumers that only use the API result, so this helps
admins detect such images. It is also for future-proofing, should any
new callers want to use only images of certain formats to error out
early.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
In almost all cases, there already is an expected format. Allow
setting a special value 'auto-detect' to opt-in to automatic format
detection by 'qemu-img' for the exceptions.
Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
This avoids auto-detection by qemu-img and so the information will be
correct with respect to the actual image format on the storage layer.
Should the image not be in the correct format, warn and try again
querying as raw, so the image is still listed. The image is present,
so it is better if it is listed and for some backwards compatibility.
The format is still returned as the matched format in such a case,
because that is how the image is treated, even if corrupt.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
This avoids auto-detection by qemu-img and so the information will be
correct with respect to the actual image format on the storage layer.
Should the image not be in the correct format, warn and try again
querying as raw, so the image is still listed. The image is present,
so it is better if it is listed and for some backwards compatibility.
The format is still returned as the matched format in such a case,
because that is how the image is treated, even if corrupt.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
By passing in the format, qemu-img will also check that the file is a
valid instance of the expected format instead of auto-detecting. The
size will also be correct when there previously was a wrong
auto-detection, but it is already only used for checking that the call
was successful.
It looks like this could also avoid wrongly failing checks with the
parent information, but a raw image cannot have a base volume and raw
images are those where the auto-detection might lead to a different
format.
Remove the unused variable $used while at it.
The check for $format in the result became obsolete with 35533c6 ("add
subvol support for directory storage").
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
This avoids auto-detection by qemu-img and so the information will be
correct with respect to the actual image format on the storage layer.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
This avoids auto-detection by qemu-img and so the information will be
correct with respect to the actual image format on the storage layer.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Make it consistent with importing, which already relies on
parse_volname() for the format.
This could cause migration failures where the format returned by
file_size_info() would not match the one from parse_volname().
Pass the format that will be used for export to file_size_info() to
ensure the correct size will be determined.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Allow callers to opt-out of 'qemu-img' autodetecting the format.
Currently not supported to be done together with untrusted, because it
can lead to less checks being done. Could be further refined (e.g.
disallow only untrusted together with format being 'raw') should the
need arise.
For 'subvol' format, the checking is handled outside of 'qemu-img' of
course, based on whether it is a directory or not.
Currently, there is a fallback to 'raw' should the format not be among
the ones allowed for the 'pve-qm-image-format' standard option. This
is to reduce potential for fallout, in particular for the plan to
change the base plugin's volume_size_info() to pass in the expected
format when calling file_size_info() too.
While not explicitly part of the storage plugin API, the 'untrusted'
parameter is now in a different place, so a compat check is added for
external plugins that might've still used it.
Breaks for qemu-server needed (if we don't want to just rely on the
compat check).
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
neither vmdk images with multiple children, nor ones with multiple extents
(that might in turn be backed by multiple files) are allowed when an image is
untrusted.
Reported-by: Friedrich Weber <f.weber@proxmox.com>
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Some OVAs have a UID/GID set for their inner file, for example the one
from GNS3:
> tar tvf 'GNS3 VM.ova' --numeric-owner
> -rw-r----- 6/1 9047 2024-11-07 10:22 GNS3 VM.ovf
> -rw-rw---- 6/1 904088064 2024-11-07 10:22 GNS3 VM-disk001.vmdk
> -rw-rw---- 6/1 2879488 2024-11-07 10:22 GNS3 VM-disk002.vmdk
As we run as root, tar is defaulting to the `--same-owner` option,
where it tries extracting files with the same ownership as exists in
the archive.
This might not be ideal and results in an error for GNS3:
> tar: GNS3 VM-disk001.vmdk: Cannot change ownership to uid 6, gid 1: Operation not permitted
So, explicitly set the `--no-same-owner` option to make tar always use
the UID/GID of the running process, which is what we want here.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Seems that some OVF do not have a ovf:Name element, but do have a
ovf:id attribute inside the ovf:VirtualSystem node that spells out
what the archive contains. So fallback to this attributes value if we
could not find any explicit name, can only win here, and the user
still can override this anyway.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This is nicer from a readability POV, but replace a arbitrary amount
of whitespace by a single minus character to avoid making it look odd.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Add a SAFE_CHAR_WITH_WHITESPACE_CLASS_RE sister variant of the shared
SAFE_CHAR_CLASS_RE shared regex to the base storage module, this use
case exist is a generic one after all, and use swap the untaint method
that parses the file a disk references to it.
Note that this is only the disk file name from inside the archive and
thus during the extraction to a staging/working directory, from there
it will be imported as volume allocated by the common storage system,
and thus follow our ordinary volume name scheme.
Improves disk detection when importing, e.g., the from upstream
provided GNS3 OVA.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Some OVFs like for example the one from the GNS3 OVA doesn't has that
namespace/prefix, and does't really hurts us to make it optional as
long as the rest is correct.
Brings us nearer to have working disks with GNS3.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This is used when finding the firmware type, so register it here or
libxml/xpath will complain about an "Undefined namespace prefix"
The schema URL was taken from some OVFs found in the wild.
Reported-by: Filip Schauer <f.schauer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
If the base image (parent) of an image contains e.g. whitespace in it's
path, the current untainting would not match and it would seem there was
no parent.
Since untrusted files are not allowed to have backing parts, just warn,
when encountering this case to keep backwards compatibility.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
introducing a separate regex that only contains ova, since
upload/downloading ovfs does not make sense (since the disks are then
missing).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
by iterating over the relevant parts and trying to parse out the
'ResourceSubType'. The content of that is not standardized, but I only
ever found examples that are compatible with vmware, meaning it's
either 'e1000', 'e1000e' or 'vmxnet3' (in various capitalizations; thus
the `lc()`)
As a fallback i used e1000, since that is our default too, and should
work for most guest operating systems.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
simply add all parsed disks to the boot order in the order we encounter
them (similar to the esxi plugin).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
it seems there is no part of the ovf standard that handles which type of
bios there is (at least i could not find it). Every ovf/ova i tested
either has no info about it, or has it in a vmware specific property
which we parse here.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
use the standards info about the ostypes to map to our own
(see comment for link to the relevant part of the dmtf schema)
every type that is not listed we map to 'other', so no need to have it
in a list.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
moves the filepath code a bit more closer to where it's actually used
checks the contained path before trying to find it's absolute path
properly add error handling to realpath
instead of checking the combined ovf_path + filepath, just make sure
filepath can't point to anythign besides a file in this directory
by checking for '.' and '..' (slashes are not allowed in SAFE_CHAR_CLASS_RE)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
since we want to handle ova files (which are only ovf+images bundled in
a tar file) for import, add code that handles that.
we introduce a valid volname for files contained in ovas like this:
storage:import/archive.ova/disk-1.vmdk
by basically treating the last part of the path as the name for the
contained disk we want.
in that case we return 'import' as type with 'vmdk/qcow2/raw' as format
(we cannot use something like 'ova+vmdk' without extending the 'format'
parsing to that for all storages/formats. This is because it runs
though a verify format check at least once)
we then provide a function to use for that:
* extract_disk_from_import_file: this actually extracts the file from
the archive. Currently only ova is supported, so the extraction with
'tar' is hardcoded, but again we can easily extend/modify that should
we need to.
we currently extract into the either the import storage or a given
target storage in the images directory so if the cleanup does not
happen, the user can still see and interact with the image via
api/cli/gui
we have to modify the `parse_ovf` a bit to handle the missing disk
images, and we parse the size out of the ovf part (since this is
informal only, it should be no problem if we cannot parse it sometimes)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
