Commit Graph

454 Commits

Author SHA1 Message Date
ccdf8ddbda LIO: followup: fix indentation 2018-08-02 14:56:20 +02:00
d9254744a8 LIO: followup: remove trailing whitespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-08-02 14:56:20 +02:00
46c6107eb1 Linux LIO/targetcli support
Introducing LIO/targetcli support allowing to use recent linux
distributions as iSCSI targets for ZFS volumes.

In order for this to work, two preconditions have to be met:

1. the portal has to be set up correctly using targetcli
2. the initiator has to be authorized to connect to the target
   based on the initiator's InitiatorName

When adding a LIO iSCSI target, a new "LIO target portal group" field needs
to be correctly populated in the "Add: ZFS over iSCSI" popup, containing the
fitting "LIO target portal group" name (typically something like 'tpg1').

Signed-Off-By: Udo Rader <udo.rader@bestsolution.at>
Tested-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2018-08-02 14:56:20 +02:00
2c2fd98b87 add metadata_size and _used to lv list
so that we can show it in the webinterface and the user can check
how full it is

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-08-02 11:16:05 +02:00
49d149e638 extend list_thinpools for multiple vgs and more information
if no vg is given, give back all thinpools from all vgs
if verbose is 1, then give back the information about the thinpools
(like size and free)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-08-02 11:10:33 +02:00
8cccb3447b add an option to include pvs in lvm_vgs
this will be used for the lvm part of the disk management

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2018-08-02 09:38:27 +02:00
074b2cb4fa remove unused Data::Dumper usages
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-11 12:23:44 +02:00
5527c824dd cephtools: simplify ceph_check_keyfile 2018-07-04 16:56:24 +02:00
5402cea50d cephfs plugin: followup with some code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-04 15:07:33 +02:00
fe0d606f5e Use keyfile create/remove from CephTools
in the RBDPlugin, that is also shared by the CephFSPlugin

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-07-04 13:18:32 +02:00
e34ce14443 Cephfs storage plugin
- ability to mount through kernel and fuse client
 - allow mount options
 - get MONs from ceph config if not in storage.cfg
 - allow the use of ceph config with fuse client
 - Delete secret on cephfs storage creation

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-07-04 13:18:19 +02:00
3e47917203 Add simple keyring check for cephfs/rbd
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-07-04 13:13:21 +02:00
9b7ba1db42 Merge RBD and CephFS code into a helper module
Some methods for connecting to a ceph cluster are the same for RBD and
CephFS, these are merged into the helper modules.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-07-04 13:13:21 +02:00
f9602323c6 lvm: use add hook
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-03 12:27:34 +02:00
2e109b4bc2 rbd: use add/delete hooks
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-03 12:27:34 +02:00
ab5e32bb1d cifs: use add/delete hooks
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-03 12:27:34 +02:00
3932ca0d1b add on_add and on_delete hooks
on_add_hook allows to encapsulate storage specific add steps, like
copying a keyring (RDB) or creating a volume group (LVM) in a clean
manner.
The same for deletion with on_delete_hook, here all should be cleaned
up, as much as possible.

Until now, this was done directly in the api config CREATE and DELETE
code, respectively, with a series of

if ($storage_type eq 'foo) {
    ...
} elsif ($storage_type eq 'bar') {
    ...
}

which isn't really that nice...

Another nice result of this approach is that also external plugins
can use those hooks and to their setup/cleanup steps sanely.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-07-03 12:27:34 +02:00
3160dbf139 CIFS: add mkdir option allowing to switch off path creation
see commit: c7616abcb2
2018-06-14 07:49:43 +02:00
5bc3edb243 followup: fix typo 2018-06-06 15:38:22 +02:00
c2f12dc648 Use enums for smbvers to prevent incorrect input 2018-06-06 15:36:25 +02:00
ff6fa67fb6 Make check_connection less costly.
This command will only check the needed share
and do not query the hole server shares.

This reduce the answer time and also has the benefit we check the
credentials on this share and not on the server.
2018-05-22 13:18:12 +02:00
9a75947b49 fix #1770: allow ex/import linked clones.
We encode the base-volume-name in the volname what is not needed.
On ex/import we need the correct dataset name.
2018-05-22 11:36:36 +02:00
840e3797a5 cifs: fix uninitialized warning in check_connection
otherwise this spams the log when check_connection runs into a timeout.
2018-05-16 10:06:00 +02:00
5fc02afb7d fix #1754: rbd: fix handling of external clusters
with the recent refactoring, external clusters were not handled
correctly with librd if a pveceph or storage specific ceph config
exists.

change the behaviour to include the pveceph config file only for pveceph
managed clusters, and a storage specific one only for external ones.

set mon_host correctly using the values from storage.cfg for external
librbd clusters.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2018-05-08 10:09:58 +02:00
fc05c9a0d8 fixup error message typo: s/sould/should/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2018-05-08 09:34:37 +02:00
cf8ded6a96 fix #1691: increase timeout in worker
A ZFS storage under heavy load can take more time.
2018-05-07 12:01:20 +02:00
894b9ecc1e fix #1691: replace udev check
`zfs create` add the creation job in a worker queue,
which should normally execute instantly. But there are circumstances
where the job will take a while to get processed.
If this is the case udev settle will see no dev in the queue and the program
will continue without an allocated dev.

The busy waiting is not best practice but the only way to be sure,
that the block device exists.
2018-05-07 12:01:20 +02:00
0b45fb3eb5 Fix #1750: set monhost split to old behavior
The path method of the RBDPlugin got a list with comma separated monhosts,
but it needs the list with semi-colon for qemu.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-05-04 14:18:04 +02:00
6cc88e8ef1 Refactor of method build_cmd and path
Method build_cmd and path use similar code to generate the ceph command
line or qemu config parameters. They now use the private method
ceph_connect_option for parameter generation.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-04-18 10:45:06 +02:00
41aacc6cde Fix #1542: show storage utilization per pool
- get storage utilization per pool
 - use librados2-perl for pool status
 - add librados2-perl as build-depends and depends in debian/control

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-04-18 10:43:14 +02:00
a9db2ca8a5 cifs: move plugin related code into plugin 2018-03-16 13:08:32 +01:00
d787523984 Register CIFSPlugin in the storage plugin system. 2018-03-16 12:06:14 +01:00
4792d439ad Add CIFS Storage Plugin.
This Plugin use as template the NFSpluigin.
We do only support smbversion 2 and 3.
Version 3 is default and must override through the config.
2018-03-16 11:57:57 +01:00
6ea1a3f3fe Fix typo in sub s/krdb_feature_disable/krbd_feature_disable
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-03-02 13:28:50 +01:00
4c3b308583 Fix #1574: could not disable krbd-incompatible image features
To prevent an error when disabling features of a rbd image with already
disabled flags. This aborted the CT/VM cloning halfway through with
a leftover rbd image, but no vmid.conf to it.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-03-02 13:28:50 +01:00
9edb99a5a7 add Storage::get_bandwidth_limit helper
Takes an operation, an optional requested bandwidth
limit override, and a list of storages involved in the
operation and lowers the requested bandwidth against global
and storage-specific limits unless the user has permissions
to change those.
This means:
 * Global limits apply to all users without Sys.Modify on /
   (as they can change datacenter.cfg options via the API).
 * Storage specific limits apply to users without
   Datastore.Allocate access on /storage/X for any involved
   storage X.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-01-31 12:25:32 +01:00
5da48ca6ea add select_existing to plugindata for iscsi(direct)
this will be used in the gui, for determining if we need to select
something from the storage when using for an image

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-11-14 13:23:52 +01:00
6b4acdb460 ISCSIPlugin: Use long options for readability
We otherwise use the long options everywhere in the plugin.
This will build the following command:

iscsiadm  --mode session --sid 1 --rescan
Rescanning session [sid: 1, target: xxx, portal: yyy]
2017-10-09 10:50:11 +02:00
79127fb5c7 rbd: support auth=none if pveceph-managed
preserve the old behaviour of selecting auth_supported based on the
existence of the keyring, but limit it to external clusters.

this allows switching 'auth XXX required' in the pveceph-managed
ceph.conf while still automatically copying the keyring when adding a
storage.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-12 11:24:39 +02:00
6eebc4a78a rbd: implement handling of unset monhost
add /etc/pve/ceph.conf to commands / option strings instead
of the monitor list provided via the 'monhost' option.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-07 11:35:00 +02:00
239aa73e93 rbd: refactor cmdline helpers
these were line by line identical except for the binary path

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-07 11:35:00 +02:00
0b9ef02ebc rbd: make monhost option optional
to allow differentiating between user-created external RBD storage
entries (WITH monhost), and those created and managed by pveceph
(without).

making monhost non-fixed allows easily opting into the managed behaviour via
'pvesm set STORAGE -delete monhost', but is also helpful for external clusters
(i.e., after adding or removing a monitor you need to update the monhost
parameter..)

adapt description accordingly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-09-07 11:35:00 +02:00
de8eff4d31 dir: allow a path as is_mountpoint value
This turns is_mountpoint more into export(5)'s `mountpoint`
property.
Given the directory storage with the properties:

    path /a/b/c
    is_mountpoint $value

$value = yes
    Same as before, /a/b/c must be mounted.
$value = no (or not set)
    Same as before, no effect.
$value = /a/b
    New: /a/b must be mounted (as opposed to /a/b/c)
2017-09-06 09:26:20 +02:00
21aaefd522 fix ceph_version_parser
Ceph change ceph version output.

full output of 'ceph --version'

Luminous 'ceph version 12.1.0 (262617c9f16c55e863693258061c5b25dea5b086) luminous (dev)'
Jewel 	 'ceph version 10.2.7 (50e863e0f4bc8f4b9e31156de690d765af245185)'
2017-07-04 06:10:15 +02:00
e53050ed13 Fixes: #1415 do not block pvestatd when nfsd is stopped on the server side
Wrap the -d test with run_or_get_killed sub this test
can make pvestatd hang on I/O wait when a nfsd process is stopped

This might help with other file based storages, for instance
directory storages on unplugged USB devices.
2017-06-29 15:16:08 +02:00
7118dd916b PVE::Storage::storage_can_replicate - hew helper 2017-06-27 06:17:58 +02:00
6b3a4a2504 import/export: drop the 'subvol/' prefix from 'tar+size'
It was deemed an unnecessary overhead.

Also add a comment about the effect (or lack thereof) of
user namespace uid mappings.
2017-06-22 17:35:04 +02:00
766cfd9a87 import/export: use our standard tar flags
Custom plugins may reuse
@PVE::Storage::Plugin::COMMON_TAR_FLAGS as well.
2017-06-22 12:47:29 +02:00
e1f6cb3904 re-add changes to PVE/Storage/Plugin.pm (fixup previous commit)
This was removed by accident.
2017-06-22 09:05:29 +02:00
364e63647a bump version to 5.0-6 2017-06-22 08:36:11 +02:00