Commit Graph

1109 Commits

Author SHA1 Message Date
609f8ec6cf LIO: Make the target cache works per target and portal
When working with several ZFS over iSCSI / LIO storages, we might lookup
between them with less than 15 sec interval.
Previously, the cache of the previous storage was used, which was breaking
disk move for example

Signed-off-by: Daniel Berteaud <daniel@firewall-services.com>
2019-09-26 18:16:42 +02:00
61137a54d6 lio: list_lun: return early if volname cannot be parsed
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-26 15:31:07 +02:00
2dbca26d99 lio: list_lun: re-use $object variable
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-26 15:30:27 +02:00
46e27c1ebc bump version to 6.0-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-23 18:20:49 +02:00
c243be83b4 Parse volname where needed
The common ZFSPlugin was missing volume name parsing
in a few places. This was not a problem for standard
volumes, but broke functionnalities (like resize,
snapshot, rollback) with linked clones as the name of
the zvol must be extracted from the entry in the config
(remove base-X-disk-Y prefix)

Signed-off-by: Daniel Berteaud <daniel@firewall-services.com>
2019-09-23 16:09:10 +02:00
b7c8738f17 Enable unmap support
In the default config, emulate_tpu is set to 0, which disables
unmap support. Once enabled, trim can run from guest to reclaim free
space.

Signed-off-by: Daniel Berteaud <daniel@firewall-services.com>
2019-09-23 16:09:10 +02:00
525ed353dc Don't remove and recreate lun when changing a volume
It's not needed, LIO sees the new size automatically.
And it was broken anyway. Partially fix #2335

Signed-off-by: Daniel Berteaud <daniel@firewall-services.com>
2019-09-23 16:09:10 +02:00
80699b1da5 followup: add JSON use and cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-20 13:32:18 +02:00
af0335e82f change file_size_info sub to use qemu-img info json decoding
Using the json output, as suggested by Thomas, we now die if the decoding
fails and, if not, all return values are set to the corresponding decoded
values. That should prevent any unforeseen null size values, except if
qemu-img info reports it, which we then consider as valid.

Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-09-20 13:01:00 +02:00
cfdffd8a20 Actually use target_volid
Migration with --targetstorage was broken because of this.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-17 09:28:52 +02:00
ce8b24a937 fix vmid filter for backup listing
$1 and $2 get set to undef from the vmid filter regex, so we have to do
the name/format regex after, else we get errors like:

'use of unitiialized value $1[...]'

and the listing is empty

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-09-09 16:38:37 +02:00
1dbbd5abd6 file_size_info: move parser to own variable
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-09 15:36:14 +02:00
aa4594b1d1 warn if filesize info couldn't be read due to error
This sh
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-09-09 15:27:26 +02:00
3918b96afa whitespace cleanup
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-09-09 15:27:26 +02:00
c844dca4a8 bump version to 6.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-03 10:24:46 +02:00
e79ab52c14 Fix #2346: rbd storage shows wrong %-usage
The patch uses the value from the field 'stored' if it is available.

In Ceph 14.2.2 the storage calculation changed to a per pool basis. This
introduced an additional field 'stored' that holds the amount of data
that has been written to the pool. While the field 'used' now has the
data after replication for the pool.

The new calculation will be used only if all OSDs are running with the
on-disk format introduced by Ceph 14.2.2.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2019-09-03 10:19:45 +02:00
b295e05ebe followup: just use case sensitive flag, no manual hack
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-03 09:42:15 +02:00
fd9ca08c70 fixup: use our to make $iso_extension_re value available for other modules
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-03 09:41:50 +02:00
4c69349140 fix #2216: Allow .img files in 'iso' type storages
To maintain full (backwards) compatibility, leave the type name as
'iso' - this makes this patch work without changing every consumer of
storage APIs.

Note that currently these files can only be attached as a CDROM/DVD
drive, so USB-only images can be uploaded but might not work in VMs.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-09-03 07:09:28 +02:00
b9993c4e9d bump version to 6.0-7
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 14:08:02 +02:00
56362cfb55 ZFS: refactor waiting for zvol symlinks
and actually do that not just for creating zvols, but also when
activating them. this should fix a range of issues/races that sometimes
occured on bootup, snapshot rollback or similar operations.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 13:39:43 +02:00
be78543909 fix 2317: list images on 'rootdir' only storages
plugins can still override list_volumes if they want separate methods to
list rootdir and images content.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 13:39:43 +02:00
4dfc2ff21e bump version to 6.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-07-25 13:35:07 +02:00
1f56f6f8d1 Fix #582: Add DELETE delay logic
Previously, the web GUI timed out when removing content (e.g. backup) took
too long. Doing the main part of the API DELETE call in a fork_worker solves
this.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2019-07-25 10:04:38 +02:00
a14e0a5e95 storage plugin: move get_subdir call
since list_volumes is only supposed to be called with filtered content
types, this should ensure that get_subdir is only called for plugins
that have a defined 'path' property, like the old code in
PVE::Storage::template_list did.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-07-19 14:04:34 +02:00
c2fc9fbe1d storage plugin: new list_volumes plugin method
This cleanup improve code reuse, and allows plugins to override list_volumes.
2019-07-19 13:49:29 +02:00
bfb3d42daf Diskmanage: add flag for encrypted osds
we can only do this here, since the ceph cluster is not aware of
osd encryption, only the local node is (via ceph-volume and lv tags)

this way, we are able to show an 'encrypted' flag in the disk gui at least

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:38:24 +02:00
1e6057d1b4 Diskmanage: fix usage test for sdd
sdd1 and sdd2 were listed in the zfs output but were not included
in the partlist, so the zfs detection test was wrong

now it is correct and more in line with the real world

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
b8cbfd362f Diskmanage: add test for lsblk by simulating zfs disk
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
8cd6d7e8e3 Diskmanage: change parttype uuid detection
previously ceph included a udev rule to populate
/dev/disk/by-parttypeuuid/

but not anymore, so we now use 'lsblk --json -o path,parttype' to
get a mapping between parttype uuid and partition

fix the test by simulating empty lsblk output

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
1c5dfef226 bump version to 6.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-07-11 19:37:31 +02:00
228e5be920 pvesm import: allow to pass a tcp://<IP> as file
To allow getting closer to finally drop "pvecm mtunnel".
Code parts taken from pipe_socket_to_command

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[regex fixup]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-11 10:13:37 +02:00
79f4a7bfd3 fix #2266: Diskmanage: get correct osd id
The osdid can consist of multiple digits, cope with that and add more
regression tests for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-08 19:04:08 +02:00
0841d128be bump version to 6.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-07-04 16:00:48 +02:00
82881c5f9e cephfs: make is_mounted check less strict
checking '$server:$subdir' is too strict to work in all cirumcstances,
e.g. adding/removing a monitor would mean that it is not the same
anymore, same if one is adding/removing the ports from the config

check only if the subdir is the same and if it is a cephfs
this way, it still returns true if someone changes the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-03 10:15:14 +02:00
35ccd3d004 fix missing osd info for osd 0
0 is falsy, we have to check for definedness
also adapt the tests so we test for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-03 08:59:47 +02:00
099bb9a389 bump version to 6.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-27 16:18:03 +02:00
5b79dac99b CephConfig: read monitor addresses also from mon_host for cephfs
since we write only the mon_host config beginning with nautilus,
we have to get the monitor ips from there as well

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-27 16:16:36 +02:00
c8a3234574 CephConfig: refactor host port parsing
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-27 16:16:36 +02:00
c835d9eca8 CephConfig: do not always interpret '; ' as a comment
; is the beginning of a comment, but in some configuration settings
it is also valid syntax, e.g. for mon_host it is a valid
seperator for hosts (sigh ...)

only remove lines when it starts with a ';'

since we remove all comments anyway any time we write the ceph conf
it should not really matter for our users

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-27 16:16:36 +02:00
b0125b6ae5 bump version to 6.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-24 18:48:27 +02:00
60fd708242 followup: comment reword
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-19 10:23:01 +02:00
9841fa31b5 CephConfig: map special config key characters to _
we want a consistent config has, regardless of how the user or a tool
adds it to the config, so we map ' ' and '-' to '_' in the keys

this way we can always access the correct key without trying multiple
times

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-19 10:21:42 +02:00
de0cd0c2e0 partially revert follow up
see
https://pve.proxmox.com/pipermail/pve-devel/2019-June/037575.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-17 12:35:56 +02:00
f6f22cb3a2 followup: free_image: just check if a file in general exist
don't require any specific file types, if something is here which can
be requested to delete over API/CLI it either comes from an API/CLI
operation and should be thus OK to delete, else a user caused the
creation of the special file and it either works and all is good or
the user gets notified as we check if unlink succeeded anyway

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-17 11:51:29 +02:00
26f14f6b07 fix #2226: allow broken symlinks to be deleted via API
Symlinks with a non-existing target fail Perls '-f' test and were thus
not deleteable via the API (failing with '$path does not exist').

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-06-17 11:49:33 +02:00
5f73686fd3 fix #2226: ignore broken symlinks in *_list
Broken symlinks (and other files without a size) will now show up as 0
byte instead of causing a format validation error in the API.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-06-17 11:49:33 +02:00
ea02de000f bump version to 6.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-14 20:44:46 +02:00
9553b55c4f cephfs: Exclude _netdev when mounting with fuse
Since ceph-fuse is called directly in the CephFS storage plugin, which
can not process the _netdev option, mounting the CephFS storage fails
when fuse is set in the storage.cfg.

This patch moves the _netdev option into the else part of the if fuse is
set statement. _netdev is only added if the CephFS kernel client mounts
the storage.

It seems _netdev is not needed anyway for the fuse mount, as the
connection is closed, once the fuse process gets killed on shutdown.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2019-06-14 10:04:51 +02:00
243f413ad2 followup: reword comment a bit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-13 11:59:58 +02:00