Commit Graph

110 Commits

Author SHA1 Message Date
80699b1da5 followup: add JSON use and cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-20 13:32:18 +02:00
af0335e82f change file_size_info sub to use qemu-img info json decoding
Using the json output, as suggested by Thomas, we now die if the decoding
fails and, if not, all return values are set to the corresponding decoded
values. That should prevent any unforeseen null size values, except if
qemu-img info reports it, which we then consider as valid.

Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-09-20 13:01:00 +02:00
ce8b24a937 fix vmid filter for backup listing
$1 and $2 get set to undef from the vmid filter regex, so we have to do
the name/format regex after, else we get errors like:

'use of unitiialized value $1[...]'

and the listing is empty

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-09-09 16:38:37 +02:00
1dbbd5abd6 file_size_info: move parser to own variable
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-09 15:36:14 +02:00
aa4594b1d1 warn if filesize info couldn't be read due to error
This sh
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-09-09 15:27:26 +02:00
3918b96afa whitespace cleanup
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-09-09 15:27:26 +02:00
4c69349140 fix #2216: Allow .img files in 'iso' type storages
To maintain full (backwards) compatibility, leave the type name as
'iso' - this makes this patch work without changing every consumer of
storage APIs.

Note that currently these files can only be attached as a CDROM/DVD
drive, so USB-only images can be uploaded but might not work in VMs.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-09-03 07:09:28 +02:00
be78543909 fix 2317: list images on 'rootdir' only storages
plugins can still override list_volumes if they want separate methods to
list rootdir and images content.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 13:39:43 +02:00
a14e0a5e95 storage plugin: move get_subdir call
since list_volumes is only supposed to be called with filtered content
types, this should ensure that get_subdir is only called for plugins
that have a defined 'path' property, like the old code in
PVE::Storage::template_list did.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-07-19 14:04:34 +02:00
c2fc9fbe1d storage plugin: new list_volumes plugin method
This cleanup improve code reuse, and allows plugins to override list_volumes.
2019-07-19 13:49:29 +02:00
de0cd0c2e0 partially revert follow up
see
https://pve.proxmox.com/pipermail/pve-devel/2019-June/037575.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-17 12:35:56 +02:00
f6f22cb3a2 followup: free_image: just check if a file in general exist
don't require any specific file types, if something is here which can
be requested to delete over API/CLI it either comes from an API/CLI
operation and should be thus OK to delete, else a user caused the
creation of the special file and it either works and all is good or
the user gets notified as we check if unlink succeeded anyway

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-17 11:51:29 +02:00
26f14f6b07 fix #2226: allow broken symlinks to be deleted via API
Symlinks with a non-existing target fail Perls '-f' test and were thus
not deleteable via the API (failing with '$path does not exist').

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-06-17 11:49:33 +02:00
11efb06659 drop un-maintained sheepdog plugin
as already announced over two months ago[0], remove the unofficial
SheepDog plugin now completely. Besides that it was never fully
supported in Proxmox VE one of its main developer and ex-maintainer
declared it as abandoned[1], and thus just let's remove it, git
allows to resurrect it any time if a wonder happens anyway.

[0]: https://pve.proxmox.com/pipermail/pve-user/2019-March/170497.html
[1]: http://lists.wpkg.org/pipermail/sheepdog/2019-March/068449.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-04 17:02:47 +02:00
d560ec2860 map_volume: fall back to 'path'
Adds a fallback to 'Plugin::path' in the default implementation of
'map_volume' to simplify a common case of calling 'map_volume' followed
by a defined-check and a call to path if it is not. The path is now
always returned if the plugin in question does not override
'map_volume'.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2019-04-29 13:44:40 +00:00
a8ec2f0227 fix #585: remove leftover disks/directory after VM creation failed
When trying to create a qcow2 disk image with a size larger than available on the
storage, this will fail.
As qemu-img does not clean up the disk afterwards, it needs to be deleted
explicitly. Further, the vmid folder is cleaned up once it is empty.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-03-05 10:36:51 +01:00
d1eb35ea74 enable snippets content type for all directory based storages
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-01-31 11:04:29 +01:00
7c7ae12f43 add new content type 'snippets'
will be used to contain files which can be executed as hookscripts or
contain custom cloud-init configs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-01-31 11:04:29 +01:00
712e27f178 Fix #1941: remove empty directories when freeing image on FS based storages
Remove directories if they are empty, which can happen if all images
from a VM got deleted, e.g., after destroying said VM.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-01-24 15:09:20 +01:00
40d698932e implement map_volume and unmap_volume
This allows to request a mapped device/path explicitly, regardles of
the storage option, eg. krbd option in the RBDplugin.

Bump of the storage ABI => 2

Co-authored-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2018-11-09 17:25:51 +01:00
dd1fa860d0 get_vm_disk_number: follow up cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-09-14 09:19:08 +02:00
f4cc2c4afe Fix #1913: get_vm_disk_number: clone plugindata to avoid side effects
Accessing a non-existing 'format' key in plugindata (e.g., in LvmThinPlugin),
created it by autovivication, thus breaking the fallback to the default value
'raw' upon any following access.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2018-09-14 09:17:40 +02:00
83a40b8d3b refactor is_valid_vm_diskname to private simpler sub
This was newly introduced and is only used once, so having a
wantarray return mechanism, without ever using it or knowing for sure
if this may help with reuse of the method is not ideal.

Make the sub a module private one just returning the vm disk number
or explicit undef. Pass it the $suffix variable, to avoid recomputing
it every time called by out caller's loop.

If there's re-use potential in the future we can actually decide what
makes sense to return.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-09-11 07:52:43 +02:00
c4a29df483 refactor finding next diskname for all plugins
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2018-09-10 12:21:10 +02:00
59fa9fd6a3 next diskname: start ids with 0 to honor MAX_VOLUMES_PER_GUEST
else we can only have MAX_VOLUMES_PER_GUEST-1 disk per VMID,
not tragic but possible confusing

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-09-10 11:21:12 +02:00
c0535aa72f make max number of disks a constant
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2018-09-07 15:28:15 +02:00
345f898108 add vm_diskname helpers (get_next, is_valid)
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2018-09-07 15:28:15 +02:00
074b2cb4fa remove unused Data::Dumper usages
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-11 12:23:44 +02:00
e34ce14443 Cephfs storage plugin
- ability to mount through kernel and fuse client
 - allow mount options
 - get MONs from ceph config if not in storage.cfg
 - allow the use of ceph config with fuse client
 - Delete secret on cephfs storage creation

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-07-04 13:18:19 +02:00
3932ca0d1b add on_add and on_delete hooks
on_add_hook allows to encapsulate storage specific add steps, like
copying a keyring (RDB) or creating a volume group (LVM) in a clean
manner.
The same for deletion with on_delete_hook, here all should be cleaned
up, as much as possible.

Until now, this was done directly in the api config CREATE and DELETE
code, respectively, with a series of

if ($storage_type eq 'foo) {
    ...
} elsif ($storage_type eq 'bar') {
    ...
}

which isn't really that nice...

Another nice result of this approach is that also external plugins
can use those hooks and to their setup/cleanup steps sanely.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-03 12:27:34 +02:00
d787523984 Register CIFSPlugin in the storage plugin system. 2018-03-16 12:06:14 +01:00
e53050ed13 Fixes: #1415 do not block pvestatd when nfsd is stopped on the server side
Wrap the -d test with run_or_get_killed sub this test
can make pvestatd hang on I/O wait when a nfsd process is stopped

This might help with other file based storages, for instance
directory storages on unplugged USB devices.
2017-06-29 15:16:08 +02:00
7118dd916b PVE::Storage::storage_can_replicate - hew helper 2017-06-27 06:17:58 +02:00
6b3a4a2504 import/export: drop the 'subvol/' prefix from 'tar+size'
It was deemed an unnecessary overhead.

Also add a comment about the effect (or lack thereof) of
user namespace uid mappings.
2017-06-22 17:35:04 +02:00
766cfd9a87 import/export: use our standard tar flags
Custom plugins may reuse
@PVE::Storage::Plugin::COMMON_TAR_FLAGS as well.
2017-06-22 12:47:29 +02:00
e1f6cb3904 re-add changes to PVE/Storage/Plugin.pm (fixup previous commit)
This was removed by accident.
2017-06-22 09:05:29 +02:00
364e63647a bump version to 5.0-6 2017-06-22 08:36:11 +02:00
9559a62a3d import/export: new formats: raw, tar, qcow2, vmdk
All of them have a `+size` prefix to show that they're not
"pure raw" or "pure tar" streams, because some storage may
need to know in advance how much storage to allocate.
The formats are explained in comments.

PVE::Storage::Plugin now has default implementations for
these for non-incremental streams exporting the current
(rather than a snapshot state).
To use qcow2 or vmdk formats $with_snapshots must be true,
otherwise raw/tar will be used where $with_snapshots must
be false.
2017-06-22 05:59:34 +02:00
636ac5b82f PVE::Storage::volume_snapshot_list - remove comment about ordering
Some storage types supports arbitrary snapshot trees, so there is
no strict ordering relation.
2017-06-07 06:36:55 +02:00
8b622c2dff PVE::Storage::volume_snapshot_list - remove $prefix parameter
Always return the full list of snapshots. Users of this library can easily
filter with a simply 'grep' instead.
2017-06-07 06:20:07 +02:00
d390328bfd api: add import/export format querying 2017-05-12 14:42:17 +02:00
47f37b5362 pvesm: import/export commands 2017-05-12 14:42:16 +02:00
17be2e9a0c volume_snapshot_list: remove $ip parameter
We want to handle ssh connections somewhere else (not inside the
storage plugins).
2017-05-10 07:02:42 +02:00
3d4949692a Revert "Include new storage function volume_send."
This reverts commit b76774e57f.
2017-05-10 06:58:44 +02:00
889d7485cb Revert "Add function volume_snapshot_delete_remote."
This reverts commit 4bd0b38f53.
2017-05-10 06:55:00 +02:00
4bd0b38f53 Add function volume_snapshot_delete_remote.
This function we need for replica to handle snapshots on remote nodes.
2017-04-28 10:05:27 +02:00
aefe82ea03 Include new storage function volume_snapshot_list.
Returns a list of snapshots (youngest snap first) form a given volid.
It is possible to use a prefix to filter the list.
2017-04-28 10:05:27 +02:00
b76774e57f Include new storage function volume_send.
If the storage backend support import and export
we can send the contend to a remote host.
2017-04-28 10:05:27 +02:00
f5451f288d remove immutable flag also for subvols on directory storage
or else the removal of such templates (with rootfs size 0) fails

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-03-10 09:21:53 +01:00
c7616abcb2 path based storages: improve the mkdir option
So far this only prevented the creation of the toplevel
directory. This does not cover all problem cases,
particularly when said directory is supposed to be a mount
point, including NFS and glusterfs beside ZFS.

The directory based storages we have already use mkpath
whenever they need to create files, and for actions on files
which are supposed to exist it's fine if it errors out.
So it should also be safe to skip the creation of standard
subdirectories in activate_storage().

Additionally NFS and glusterfs storages should also accept
the mkdir option as they otherwise may exhibit similar
issues, eg. when an NFS storage is mounted onto a directory
inside a ZFS subvolume.
2016-09-27 09:54:53 +02:00