Some methods for connecting to a ceph cluster are the same for RBD and
CephFS, these are merged into the helper modules.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
with the recent refactoring, external clusters were not handled
correctly with librd if a pveceph or storage specific ceph config
exists.
change the behaviour to include the pveceph config file only for pveceph
managed clusters, and a storage specific one only for external ones.
set mon_host correctly using the values from storage.cfg for external
librbd clusters.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
The path method of the RBDPlugin got a list with comma separated monhosts,
but it needs the list with semi-colon for qemu.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Method build_cmd and path use similar code to generate the ceph command
line or qemu config parameters. They now use the private method
ceph_connect_option for parameter generation.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
- get storage utilization per pool
- use librados2-perl for pool status
- add librados2-perl as build-depends and depends in debian/control
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
To prevent an error when disabling features of a rbd image with already
disabled flags. This aborted the CT/VM cloning halfway through with
a leftover rbd image, but no vmid.conf to it.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
Takes an operation, an optional requested bandwidth
limit override, and a list of storages involved in the
operation and lowers the requested bandwidth against global
and storage-specific limits unless the user has permissions
to change those.
This means:
* Global limits apply to all users without Sys.Modify on /
(as they can change datacenter.cfg options via the API).
* Storage specific limits apply to users without
Datastore.Allocate access on /storage/X for any involved
storage X.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
preserve the old behaviour of selecting auth_supported based on the
existence of the keyring, but limit it to external clusters.
this allows switching 'auth XXX required' in the pveceph-managed
ceph.conf while still automatically copying the keyring when adding a
storage.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
add /etc/pve/ceph.conf to commands / option strings instead
of the monitor list provided via the 'monhost' option.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
to allow differentiating between user-created external RBD storage
entries (WITH monhost), and those created and managed by pveceph
(without).
making monhost non-fixed allows easily opting into the managed behaviour via
'pvesm set STORAGE -delete monhost', but is also helpful for external clusters
(i.e., after adding or removing a monitor you need to update the monhost
parameter..)
adapt description accordingly.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Ceph change ceph version output.
full output of 'ceph --version'
Luminous 'ceph version 12.1.0 (262617c9f16c55e863693258061c5b25dea5b086) luminous (dev)'
Jewel 'ceph version 10.2.7 (50e863e0f4bc8f4b9e31156de690d765af245185)'
since we allow vm-ID-whatever when allocating images, we
should also include those when listing them.
note: '@' is reserved for snapshots in ceph, so it is safe to
skip lines including an '@' in the image name.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
with more than a few images, 'rbd ls -l' gets rather slow
compared to a simple 'rbd ls'. since we only need to check
existing image names for finding a free one, the latter is
sufficient.
example with ~400 rbd images:
$ time rbd ls -p ceph-vm > /dev/null
real 0m0.027s
user 0m0.012s
sys 0m0.008s
$ time rbd ls -l -p ceph-vm > /dev/null
real 0m5.250s
user 0m1.632s
sys 0m0.584s
a linked clone of two disks on the same setup accordingly
also shows a massive speedup:
$ time qm clone 1000 10000 -snap test
create linked clone of drive scsi0 (ceph-vm:vm-1000-disk-2)
clone vm-1000-disk-2: vm-1000-disk-2 snapname test to
vm-10000-disk-1
create linked clone of drive scsi1 (ceph-vm:vm-1000-disk-1)
clone vm-1000-disk-1: vm-1000-disk-1 snapname test to
vm-10000-disk-2
real 0m11.157s
user 0m3.752s
sys 0m1.308s
$ time qm clone 1000 10000 -snap test
create linked clone of drive scsi1 (ceph-vm:vm-1000-disk-1)
clone vm-1000-disk-1: vm-1000-disk-1 snapname test to
vm-10000-disk-1
create linked clone of drive scsi0 (ceph-vm:vm-1000-disk-2)
clone vm-1000-disk-2: vm-1000-disk-2 snapname test to
vm-10000-disk-2
real 0m0.872s
user 0m0.652s
sys 0m0.096s
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
With krbd we resize volume and tell QemuSever to notify running QEMU
with zero $size by returning undef.
Signed-off-by: Dmitry Petuhov <mityapetuhov@gmail.com>
without this, having an efidisk on a ceph storage
prevents creating another disk on the same
ceph storage, because it will not be detected
and we try to allocate one with the same name
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
since the rbd images themselves are named differently than
the volumes in our config files, we need to recreate this
information from the parent relation in the ceph metadata,
otherwise list_images() might return wrong volume names/IDs
since list_images is used by PVE::Storage::vdisk_free() to
check for children still referencing a base image, because
of the wrong volume id RBDPlugin->parse_volname() does not
detect the base image of linked clones and the check fails.
this is thankfully mitigated by the protected status of the
base snapshot, but creates a rather confusing error message.
scenario (VM 701 is a linked clone of template VM 700):
$ qm config 700 | grep virtio0:
virtio0: ceph_qemu:base-700-disk-1,size=2G
$ qm config 701 | grep virtio0:
virtio0: ceph_qemu:base-700-disk-1/vm-701-disk-1,size=2G
before (pvesm list reports wrong volume ID, check fails):
$ pvesm list ceph_qemu
ceph_qemu:base-700-disk-1 raw 2147483648 700
ceph_qemu:vm-701-disk-1 raw 2147483648 701
$ pvesm free ceph_qemu:base-700-disk-1
snap_unprotect: can't unprotect; at least 1 child(ren) in pool rbd
rbd unprotect base-700-disk-1 snap '__base__' error: snap_unprotect: can't unprotect; at least 1 child(ren) in pool rbd
after (correct volume ID, check works as intended):
$ pvesm list ceph_qemu
ceph_qemu:base-700-disk-1 raw 2147483648 700
ceph_qemu:base-700-disk-1/vm-701-disk-1 raw 2147483648 701
$ pvesm free ceph_qemu:base-700-disk-1
base volume 'base-700-disk-1' is still in use (use by 'base-700-disk-1/vm-701-disk-1')
If you want to use different ceph storage,
something they have differents values like ms_nocrc = true.(they are also others ones).
The client need to specify theses special options to be able to connect
This patch allow to create a ceph config file for each storeid in
/etc/pve/priv/ceph/$storeid.conf
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
This way we get parameter verification on monitor addresses
as well as the ability to pass multiple `--monhost`
arguments to `pvesm add`.
Since our '-list' schemas default to using commas we now
need to properly support these, so all uses of the monhost
property now replace all of semicolon, space or comma into
the currently required character.
This should fix the issues reported by Alwin Antreich on the
pve-user list.
Since this schema supports both ipv6+port notations we need
to make sure we convert to the bracket enclosed variant.
Added a helper for this.
"ceph version" retrieves the version from the cluster (i.e.,
from the queried monitor), but what is needed here is the
local ceph version, which is returned by "ceph --version".
otherwise mapping those images will fail. disabling the
features only needs to be done once per image, so it makes
sense to do this when creating the images.
unfortunately, the command does not work in hammer, so
it needs a version check for jewel or higher.
This makes no sense because it should always be exclusive.
Also RDB checks it self.
LVM has not possibility to use lvchange.
DRBD is this feature not implemented.
