Commit Graph

422 Commits

Author SHA1 Message Date
4792d439ad Add CIFS Storage Plugin.
This Plugin use as template the NFSpluigin.
We do only support smbversion 2 and 3.
Version 3 is default and must override through the config.
2018-03-16 11:57:57 +01:00
6ea1a3f3fe Fix typo in sub s/krdb_feature_disable/krbd_feature_disable
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-03-02 13:28:50 +01:00
4c3b308583 Fix #1574: could not disable krbd-incompatible image features
To prevent an error when disabling features of a rbd image with already
disabled flags. This aborted the CT/VM cloning halfway through with
a leftover rbd image, but no vmid.conf to it.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-03-02 13:28:50 +01:00
9edb99a5a7 add Storage::get_bandwidth_limit helper
Takes an operation, an optional requested bandwidth
limit override, and a list of storages involved in the
operation and lowers the requested bandwidth against global
and storage-specific limits unless the user has permissions
to change those.
This means:
 * Global limits apply to all users without Sys.Modify on /
   (as they can change datacenter.cfg options via the API).
 * Storage specific limits apply to users without
   Datastore.Allocate access on /storage/X for any involved
   storage X.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-01-31 12:25:32 +01:00
5da48ca6ea add select_existing to plugindata for iscsi(direct)
this will be used in the gui, for determining if we need to select
something from the storage when using for an image

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-11-14 13:23:52 +01:00
6b4acdb460 ISCSIPlugin: Use long options for readability
We otherwise use the long options everywhere in the plugin.
This will build the following command:

iscsiadm  --mode session --sid 1 --rescan
Rescanning session [sid: 1, target: xxx, portal: yyy]
2017-10-09 10:50:11 +02:00
79127fb5c7 rbd: support auth=none if pveceph-managed
preserve the old behaviour of selecting auth_supported based on the
existence of the keyring, but limit it to external clusters.

this allows switching 'auth XXX required' in the pveceph-managed
ceph.conf while still automatically copying the keyring when adding a
storage.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-12 11:24:39 +02:00
6eebc4a78a rbd: implement handling of unset monhost
add /etc/pve/ceph.conf to commands / option strings instead
of the monitor list provided via the 'monhost' option.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-07 11:35:00 +02:00
239aa73e93 rbd: refactor cmdline helpers
these were line by line identical except for the binary path

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-07 11:35:00 +02:00
0b9ef02ebc rbd: make monhost option optional
to allow differentiating between user-created external RBD storage
entries (WITH monhost), and those created and managed by pveceph
(without).

making monhost non-fixed allows easily opting into the managed behaviour via
'pvesm set STORAGE -delete monhost', but is also helpful for external clusters
(i.e., after adding or removing a monitor you need to update the monhost
parameter..)

adapt description accordingly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-07 11:35:00 +02:00
de8eff4d31 dir: allow a path as is_mountpoint value
This turns is_mountpoint more into export(5)'s `mountpoint`
property.
Given the directory storage with the properties:

    path /a/b/c
    is_mountpoint $value

$value = yes
    Same as before, /a/b/c must be mounted.
$value = no (or not set)
    Same as before, no effect.
$value = /a/b
    New: /a/b must be mounted (as opposed to /a/b/c)
2017-09-06 09:26:20 +02:00
21aaefd522 fix ceph_version_parser
Ceph change ceph version output.

full output of 'ceph --version'

Luminous 'ceph version 12.1.0 (262617c9f16c55e863693258061c5b25dea5b086) luminous (dev)'
Jewel 	 'ceph version 10.2.7 (50e863e0f4bc8f4b9e31156de690d765af245185)'
2017-07-04 06:10:15 +02:00
e53050ed13 Fixes: #1415 do not block pvestatd when nfsd is stopped on the server side
Wrap the -d test with run_or_get_killed sub this test
can make pvestatd hang on I/O wait when a nfsd process is stopped

This might help with other file based storages, for instance
directory storages on unplugged USB devices.
2017-06-29 15:16:08 +02:00
7118dd916b PVE::Storage::storage_can_replicate - hew helper 2017-06-27 06:17:58 +02:00
6b3a4a2504 import/export: drop the 'subvol/' prefix from 'tar+size'
It was deemed an unnecessary overhead.

Also add a comment about the effect (or lack thereof) of
user namespace uid mappings.
2017-06-22 17:35:04 +02:00
766cfd9a87 import/export: use our standard tar flags
Custom plugins may reuse
@PVE::Storage::Plugin::COMMON_TAR_FLAGS as well.
2017-06-22 12:47:29 +02:00
e1f6cb3904 re-add changes to PVE/Storage/Plugin.pm (fixup previous commit)
This was removed by accident.
2017-06-22 09:05:29 +02:00
364e63647a bump version to 5.0-6 2017-06-22 08:36:11 +02:00
5cbbc78fdb lvm: import/export: implement raw+size format 2017-06-22 06:48:09 +02:00
9559a62a3d import/export: new formats: raw, tar, qcow2, vmdk
All of them have a `+size` prefix to show that they're not
"pure raw" or "pure tar" streams, because some storage may
need to know in advance how much storage to allocate.
The formats are explained in comments.

PVE::Storage::Plugin now has default implementations for
these for non-incremental streams exporting the current
(rather than a snapshot state).
To use qcow2 or vmdk formats $with_snapshots must be true,
otherwise raw/tar will be used where $with_snapshots must
be false.
2017-06-22 05:59:34 +02:00
b43d0f3043 ZFSPoolPlugin.pm: remove unused code 2017-06-08 08:45:22 +02:00
636ac5b82f PVE::Storage::volume_snapshot_list - remove comment about ordering
Some storage types supports arbitrary snapshot trees, so there is
no strict ordering relation.
2017-06-07 06:36:55 +02:00
8b622c2dff PVE::Storage::volume_snapshot_list - remove $prefix parameter
Always return the full list of snapshots. Users of this library can easily
filter with a simply 'grep' instead.
2017-06-07 06:20:07 +02:00
a3f38a644c fix #1379: return size as number instead of string
this caused the webinterface to sort alphabetically instead of numerical
when sorting by image size

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 10:24:30 +02:00
d390328bfd api: add import/export format querying 2017-05-12 14:42:17 +02:00
47f37b5362 pvesm: import/export commands 2017-05-12 14:42:16 +02:00
17be2e9a0c volume_snapshot_list: remove $ip parameter
We want to handle ssh connections somewhere else (not inside the
storage plugins).
2017-05-10 07:02:42 +02:00
3d4949692a Revert "Include new storage function volume_send."
This reverts commit b76774e57f.
2017-05-10 06:58:44 +02:00
44257d2e38 Revert "Add ip parameter in zfs_request to execute on remote host."
This reverts commit c4bb4a3d19.
2017-05-10 06:55:42 +02:00
889d7485cb Revert "Add function volume_snapshot_delete_remote."
This reverts commit 4bd0b38f53.
2017-05-10 06:55:00 +02:00
f189504ccb Add replicate as new storage feature.
This feature shows that the storage can send and receive images.
2017-04-28 10:05:27 +02:00
4bd0b38f53 Add function volume_snapshot_delete_remote.
This function we need for replica to handle snapshots on remote nodes.
2017-04-28 10:05:27 +02:00
c4bb4a3d19 Add ip parameter in zfs_request to execute on remote host.
We need this function to delete remote snapshots.
2017-04-28 10:05:27 +02:00
aefe82ea03 Include new storage function volume_snapshot_list.
Returns a list of snapshots (youngest snap first) form a given volid.
It is possible to use a prefix to filter the list.
2017-04-28 10:05:27 +02:00
b76774e57f Include new storage function volume_send.
If the storage backend support import and export
we can send the contend to a remote host.
2017-04-28 10:05:27 +02:00
c59a0f2452 rbd: fix rados df parser for luminous 2017-03-21 10:13:02 +01:00
f5451f288d remove immutable flag also for subvols on directory storage
or else the removal of such templates (with rootfs size 0) fails

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-03-10 09:21:53 +01:00
d746da86d8 rbd: fix exit from 'rbd ls -l' parser 2017-02-23 09:22:57 +01:00
b2e430d30f rbd: minor regex fixup 2017-02-20 10:00:57 +01:00
292a33fdd4 rbd: fix typo
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-02-13 12:13:06 +01:00
4db25fa2a6 rbd: use consistent image name schemes
since we allow vm-ID-whatever when allocating images, we
should also include those when listing them.

note: '@' is reserved for snapshots in ceph, so it is safe to
skip lines including an '@' in the image name.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-02-13 12:13:06 +01:00
53a236f2c5 rbd: use 'rbd ls' without '-l' to find free names
with more than a few images, 'rbd ls -l' gets rather slow
compared to a simple 'rbd ls'. since we only need to check
existing image names for finding a free one, the latter is
sufficient.

example with ~400 rbd images:
$ time rbd ls -p ceph-vm > /dev/null
real    0m0.027s
user    0m0.012s
sys     0m0.008s

$ time rbd ls -l -p ceph-vm > /dev/null
real    0m5.250s
user    0m1.632s
sys     0m0.584s

a linked clone of two disks on the same setup accordingly
also shows a massive speedup:

$ time qm clone 1000 10000 -snap test
create linked clone of drive scsi0 (ceph-vm:vm-1000-disk-2)
clone vm-1000-disk-2: vm-1000-disk-2 snapname test to
vm-10000-disk-1
create linked clone of drive scsi1 (ceph-vm:vm-1000-disk-1)
clone vm-1000-disk-1: vm-1000-disk-1 snapname test to
vm-10000-disk-2

real    0m11.157s
user    0m3.752s
sys     0m1.308s

$ time qm clone 1000 10000 -snap test
create linked clone of drive scsi1 (ceph-vm:vm-1000-disk-1)
clone vm-1000-disk-1: vm-1000-disk-1 snapname test to
vm-10000-disk-1
create linked clone of drive scsi0 (ceph-vm:vm-1000-disk-2)
clone vm-1000-disk-2: vm-1000-disk-2 snapname test to
vm-10000-disk-2

real    0m0.872s
user    0m0.652s
sys     0m0.096s

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-02-13 12:13:06 +01:00
ef881e10eb fixes for new PVE::RPCEnvironment implementation
Use PVE::RPCEnvironment->is_worker() instead of
PVE::RPCEnvironment::is_worker().
2017-01-19 09:14:41 +01:00
e438d0940f fix #1252: rbd: delete snapshots when using krbd 2017-01-17 10:32:23 +01:00
a4aee43380 Fix RBD resize with krbd option enabled.
With krbd we resize volume and tell QemuSever to notify running QEMU
with zero $size by returning undef.

Signed-off-by: Dmitry Petuhov <mityapetuhov@gmail.com>
2017-01-16 09:14:00 +01:00
e2e74320b7 sheepdog : volume_resize return if running
fix : https://forum.proxmox.com/threads/sheepdog-disk-resize-not-working.31760/

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2017-01-09 13:27:11 +01:00
2ff3b506a1 use qemu gluster blockdriver for linked clone creation
this works around a bug, where qemu does not align the qcow2 file
when using the filesystem directly, and the gluster blockdriver
refuses to read from it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-11-29 09:40:10 +01:00
72bdeea1bf increase default timeout for zpool import
as zpool import can easily take longer than 5 seconds on
systems with lots of disks
2016-11-29 09:30:26 +01:00
e2e6380112 improve zpool activate_storage
the old code was way too broad here, this fixes at least the
following issues:
- importing of other/unconfigured zpools by "import -a"
- possible false positives if a pool name is a substring of
  another pool name because of "list" without pool name,
  potentially skipping activation for such pools
- not noticing failure to activate in activate_storage
  because the success of "zpool import -a" does not tell us
  anything about the pool we actually wanted to import

checking specifically for the pool to be activated when
calling "zpool list" gets rid of the second issue, and
trying to import only that pool fixes the other two.
2016-11-29 09:29:33 +01:00
4b7dd9d743 allow --allow-shrink on RBD resize
Signed-off-by: Stefan Priebe <s.priebe@profihost.ag>
2016-11-17 09:06:55 +01:00