Adds a fallback to 'Plugin::path' in the default implementation of
'map_volume' to simplify a common case of calling 'map_volume' followed
by a defined-check and a call to path if it is not. The path is now
always returned if the plugin in question does not override
'map_volume'.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
The underlying issue is that a zpool can get imported only once, so
we first check if it's in `zpool list`, and thus imported, and only
if it does not shows up there we try to import it.
But, this can race with either:
* parallel running activate_storage call, through CLI/API/daemon
* a zpool import from an admin (a bit unlikely, but hey that's the
thing with race conditions ;))
So refactor the "is pool imported" check into a closure, and call it
addditionally if the import failed, and silent the error if the pool
is now listed, and thus imported. This makes it a little bit nicer to
read too, IMO.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
during storage activation.
for pools that don't get imported at boot (e.g. because their vdevs are
not available when zfs-import-*.service runs) it is fatal to include
them in the cachefile, for those that do get imported at boot this code
should never run anyway as they are already imported.
in any case, a fallback to import without cachefile is the safe variant.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
fixes the 'cannot create 'nvme/foo': volume size must be a multiple of
volume block size' error by always rounding the size up to the next 1M
boundary. this is a workaround until
https://github.com/zfsonlinux/zfs/issues/8541 is solved.
the current manpage says 128k is the maximum blocksize, but a local test
showed that values up to 1M are allowed. it might be possible to
increase it even further (see f1512ee61).
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
When trying to create a qcow2 disk image with a size larger than available on the
storage, this will fail.
As qemu-img does not clean up the disk afterwards, it needs to be deleted
explicitly. Further, the vmid folder is cleaned up once it is empty.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
will be used to contain files which can be executed as hookscripts or
contain custom cloud-init configs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
use the port where the main glusterfs daemon listens on as ping port,
this one is also used by QEMU as default.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Remove directories if they are empty, which can happen if all images
from a VM got deleted, e.g., after destroying said VM.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
LVMPlugin->volume_import (used by storage_migrate on either offline
migration with local disks, or online migration with storage-only
referenced disks) passed 'conv=sparse' to `dd`. This can lead to
data-corruption, if the target volume is not zero-initialized.
dropping the sparse argument completely would fix the problem, but
breaks keeping data sparse for LvmThinPlugin.
This patch moves the dd out into (LVM*) plugin specific sub so that
each can control the parameters.
Steps for reproducing the issue:
* create a cluster with (at least) 2 nodes A and B, with a free
disk-device (/dev/sdx)
* write a recognizable pattern to /dev/sdx on B:
`dd if=/dev/zero bs=10M | tr '\000' '\255' | dd of=/dev/sdb bs=10M`
(would be grateful for alternatives to the dd| tr| dd)
* on both A and B create a lvm-vg (pvcreate, vgcreate)
* add it as _not_ shared storage, which is available on nodes A and B
* create a small guest on A
* fill a file in the guest with zeros
`dd if=/dev/zero of=/zerofil bs=10M`
* stop the guest, migrate it to B
* start the guest - check that the file `/zerofil` contains `ad`
instead of `00`
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
From `man 8 lvchange`:
--refresh
If the logical volume is active, reload its metadata. This is not
necessary in normal operation, but may be useful ... if you're doing
clustering manually without a clustered lock manager.
Fixes migration in a shared LVM (iscsi) setup, where a disk gets resized on one
node A and the guest is afterwards migrated to another node B: B still presents
the old size to the guest, leading to data corruption.
It is necessary to run `lvchange` twice because the options `-ay` and
`--refresh` are mutually exclusive.
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Without volume_size_info a Storage plugin falls back to the Implementation
in PVE/Storage/Plugin.pm, which relies on `qemu-img info`.
`qemu-img info` returns wrong results on a node in the case of shared volume
groups (e.g. when sharing disks via iSCSI), if a disk was resized on another
node (it lseeks to the end of the block-device, and this yields the old size).
Using lvs directly fixes the issue, since the LVM metadata gets updated when
invoking lvs.
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
it is not a storage plugin, and it makes more sense to have it
top-level, but there we cannot name it CephTools because of the
existing ones in pve-manager
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
As we mount this manually and thus systemd doesn't know about any
dependency for cephFS mounts, this got umounted only at the last
stage of shutdown, where network wasn't active anymore.
But, CephFS needs to be connected to an active MDS for a clean
unmount so without network this mount would delay shutdown for quite
a bit, until after some minutes systemd gave up and forced unmount.
So tell systemd that this mount requires network, which can be done
with the '_netdev'[0] mount option, that lucky for us can be also
passed to a mount call and isn't only available for fstab.
with this a mount gets, among others:
> Wants=network-online.target
> Before=umount.target remote-fs.target
> After=remote-fs-pre.target system.slice network.target network-online.target -.mount
Which does the trick for us.
[0]: https://www.freedesktop.org/software/systemd/man/systemd.mount.html#_netdev
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This is important for shared LVM storages. As with deletes and
creates of images, as else we may have not the up-to-date metadata
and extents may get reused if another node created an image during
the same time, for example.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This allows to request a mapped device/path explicitly, regardles of
the storage option, eg. krbd option in the RBDplugin.
Bump of the storage ABI => 2
Co-authored-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Accessing a non-existing 'format' key in plugindata (e.g., in LvmThinPlugin),
created it by autovivication, thus breaking the fallback to the default value
'raw' upon any following access.
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
$features is actually an array reference, so use it as one.
This broke creation and migration of disks on rbd storages
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This was newly introduced and is only used once, so having a
wantarray return mechanism, without ever using it or knowing for sure
if this may help with reuse of the method is not ideal.
Make the sub a module private one just returning the vm disk number
or explicit undef. Pass it the $suffix variable, to avoid recomputing
it every time called by out caller's loop.
If there's re-use potential in the future we can actually decide what
makes sense to return.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
else we can only have MAX_VOLUMES_PER_GUEST-1 disk per VMID,
not tragic but possible confusing
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Non conforming image names are not ignored anymore by the new rbd_ls
implementation, this patch adds the old behaviour.
This fix is a temporary workaround and should be removed, once the new
image name parser is ready.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
since ceph changed the plain output format for 12.2.8
we have to change the code anyway, and when were at it,
we can change it to the (hopefully) more robust json output
Co-authored-by: Alwin Antreich <a.antreich@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
lvm_find_free_diskname only checked for existing volumes starting with 'vm-',
and not with 'base-'.
Unify implementation with other Plugins.
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
move two loop bodies from
if (condition) {
...
}
too
next if !condition;
...
to save an indentation level
rename variables to a bit shorter version, i.e.:
s/oneTarget/target/
s/oneTpg/tpg/
and a comment rewording
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Introducing LIO/targetcli support allowing to use recent linux
distributions as iSCSI targets for ZFS volumes.
In order for this to work, two preconditions have to be met:
1. the portal has to be set up correctly using targetcli
2. the initiator has to be authorized to connect to the target
based on the initiator's InitiatorName
When adding a LIO iSCSI target, a new "LIO target portal group" field needs
to be correctly populated in the "Add: ZFS over iSCSI" popup, containing the
fitting "LIO target portal group" name (typically something like 'tpg1').
Signed-Off-By: Udo Rader <udo.rader@bestsolution.at>
Tested-by: Stoiko Ivanov <s.ivanov@proxmox.com>
if no vg is given, give back all thinpools from all vgs
if verbose is 1, then give back the information about the thinpools
(like size and free)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
- ability to mount through kernel and fuse client
- allow mount options
- get MONs from ceph config if not in storage.cfg
- allow the use of ceph config with fuse client
- Delete secret on cephfs storage creation
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
