Commit Graph

1090 Commits

Author SHA1 Message Date
b9993c4e9d bump version to 6.0-7
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 14:08:02 +02:00
56362cfb55 ZFS: refactor waiting for zvol symlinks
and actually do that not just for creating zvols, but also when
activating them. this should fix a range of issues/races that sometimes
occured on bootup, snapshot rollback or similar operations.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 13:39:43 +02:00
be78543909 fix 2317: list images on 'rootdir' only storages
plugins can still override list_volumes if they want separate methods to
list rootdir and images content.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 13:39:43 +02:00
4dfc2ff21e bump version to 6.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-07-25 13:35:07 +02:00
1f56f6f8d1 Fix #582: Add DELETE delay logic
Previously, the web GUI timed out when removing content (e.g. backup) took
too long. Doing the main part of the API DELETE call in a fork_worker solves
this.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2019-07-25 10:04:38 +02:00
a14e0a5e95 storage plugin: move get_subdir call
since list_volumes is only supposed to be called with filtered content
types, this should ensure that get_subdir is only called for plugins
that have a defined 'path' property, like the old code in
PVE::Storage::template_list did.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-07-19 14:04:34 +02:00
c2fc9fbe1d storage plugin: new list_volumes plugin method
This cleanup improve code reuse, and allows plugins to override list_volumes.
2019-07-19 13:49:29 +02:00
bfb3d42daf Diskmanage: add flag for encrypted osds
we can only do this here, since the ceph cluster is not aware of
osd encryption, only the local node is (via ceph-volume and lv tags)

this way, we are able to show an 'encrypted' flag in the disk gui at least

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:38:24 +02:00
1e6057d1b4 Diskmanage: fix usage test for sdd
sdd1 and sdd2 were listed in the zfs output but were not included
in the partlist, so the zfs detection test was wrong

now it is correct and more in line with the real world

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
b8cbfd362f Diskmanage: add test for lsblk by simulating zfs disk
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
8cd6d7e8e3 Diskmanage: change parttype uuid detection
previously ceph included a udev rule to populate
/dev/disk/by-parttypeuuid/

but not anymore, so we now use 'lsblk --json -o path,parttype' to
get a mapping between parttype uuid and partition

fix the test by simulating empty lsblk output

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
1c5dfef226 bump version to 6.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-07-11 19:37:31 +02:00
228e5be920 pvesm import: allow to pass a tcp://<IP> as file
To allow getting closer to finally drop "pvecm mtunnel".
Code parts taken from pipe_socket_to_command

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[regex fixup]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-11 10:13:37 +02:00
79f4a7bfd3 fix #2266: Diskmanage: get correct osd id
The osdid can consist of multiple digits, cope with that and add more
regression tests for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-08 19:04:08 +02:00
0841d128be bump version to 6.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-07-04 16:00:48 +02:00
82881c5f9e cephfs: make is_mounted check less strict
checking '$server:$subdir' is too strict to work in all cirumcstances,
e.g. adding/removing a monitor would mean that it is not the same
anymore, same if one is adding/removing the ports from the config

check only if the subdir is the same and if it is a cephfs
this way, it still returns true if someone changes the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-03 10:15:14 +02:00
35ccd3d004 fix missing osd info for osd 0
0 is falsy, we have to check for definedness
also adapt the tests so we test for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-03 08:59:47 +02:00
099bb9a389 bump version to 6.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-27 16:18:03 +02:00
5b79dac99b CephConfig: read monitor addresses also from mon_host for cephfs
since we write only the mon_host config beginning with nautilus,
we have to get the monitor ips from there as well

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-27 16:16:36 +02:00
c8a3234574 CephConfig: refactor host port parsing
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-27 16:16:36 +02:00
c835d9eca8 CephConfig: do not always interpret '; ' as a comment
; is the beginning of a comment, but in some configuration settings
it is also valid syntax, e.g. for mon_host it is a valid
seperator for hosts (sigh ...)

only remove lines when it starts with a ';'

since we remove all comments anyway any time we write the ceph conf
it should not really matter for our users

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-27 16:16:36 +02:00
b0125b6ae5 bump version to 6.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-24 18:48:27 +02:00
60fd708242 followup: comment reword
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-19 10:23:01 +02:00
9841fa31b5 CephConfig: map special config key characters to _
we want a consistent config has, regardless of how the user or a tool
adds it to the config, so we map ' ' and '-' to '_' in the keys

this way we can always access the correct key without trying multiple
times

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-19 10:21:42 +02:00
de0cd0c2e0 partially revert follow up
see
https://pve.proxmox.com/pipermail/pve-devel/2019-June/037575.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-17 12:35:56 +02:00
f6f22cb3a2 followup: free_image: just check if a file in general exist
don't require any specific file types, if something is here which can
be requested to delete over API/CLI it either comes from an API/CLI
operation and should be thus OK to delete, else a user caused the
creation of the special file and it either works and all is good or
the user gets notified as we check if unlink succeeded anyway

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-17 11:51:29 +02:00
26f14f6b07 fix #2226: allow broken symlinks to be deleted via API
Symlinks with a non-existing target fail Perls '-f' test and were thus
not deleteable via the API (failing with '$path does not exist').

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-06-17 11:49:33 +02:00
5f73686fd3 fix #2226: ignore broken symlinks in *_list
Broken symlinks (and other files without a size) will now show up as 0
byte instead of causing a format validation error in the API.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-06-17 11:49:33 +02:00
ea02de000f bump version to 6.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-14 20:44:46 +02:00
9553b55c4f cephfs: Exclude _netdev when mounting with fuse
Since ceph-fuse is called directly in the CephFS storage plugin, which
can not process the _netdev option, mounting the CephFS storage fails
when fuse is set in the storage.cfg.

This patch moves the _netdev option into the else part of the if fuse is
set statement. _netdev is only added if the CephFS kernel client mounts
the storage.

It seems _netdev is not needed anyway for the fuse mount, as the
connection is closed, once the fuse process gets killed on shutdown.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2019-06-14 10:04:51 +02:00
243f413ad2 followup: reword comment a bit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-13 11:59:58 +02:00
5c0720d6c9 fix #1427: Set file mode on uploaded templates/ISOs
simply chmod the temp file before copying to the "correct" permission
mode, where all users with access to the directory can read the file,
to mirror the behavior one gets for a apl_download call.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-06-13 11:55:18 +02:00
11efb06659 drop un-maintained sheepdog plugin
as already announced over two months ago[0], remove the unofficial
SheepDog plugin now completely. Besides that it was never fully
supported in Proxmox VE one of its main developer and ex-maintainer
declared it as abandoned[1], and thus just let's remove it, git
allows to resurrect it any time if a wonder happens anyway.

[0]: https://pve.proxmox.com/pipermail/pve-user/2019-March/170497.html
[1]: http://lists.wpkg.org/pipermail/sheepdog/2019-March/068449.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-04 17:02:47 +02:00
3ed611874e buildsys: split storage plugins to single lines
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-04 16:56:46 +02:00
9250ddfe3a Diskmanage: correctly add wearout value of 0
if wearout is 0 we showed 'N/A' instead of 100%
(wearout is really the 'life left' value)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 13:14:45 +02:00
c5fa45a99f Diskmanage: fix incorrect variable usage
sysdir is a string, we wanted sysdata

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 13:14:45 +02:00
ea928fd41a Diskmanage: extract nvme wearout from smartctl text
extract the info from the line:
Percentage Used: XX%

also adapt the tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 13:14:45 +02:00
d0a217f624 tests: test get_disks single/multi disk filter
this executes all tests again with each disk as a single parameter
and all disks again as an array ref

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 12:51:44 +02:00
1dc3038d40 Diskmanage: add append_partition sub
we will use this for adding a partition to a disk when using a device
for ceph osd db/wal which already has partitions on it

first we search for the highest partition number, then add the partition
and search for the resulting device (we cannot assume to simply
append the number, e.g. from /dev/nvme0n1 we get /dev/nvme0n1pX)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 12:51:44 +02:00
52a064afcd Diskmanage: allow get_disks to take multiple disks
we now expect the first parameter to be either a string with a single
disk, or an array ref with a list of disks

this way we can get the info of multiple disks simultaneously while
not iterating over all disks

this will be used to get the info for osd/db/wal disk

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 12:51:44 +02:00
558d412d10 Disks API: allow also unused disks for db/wal
since we will create a pv/vg/lv on it with nautilus

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 12:51:44 +02:00
b12883ded1 tests: improve error handling of run_disk_tests
if Diskmanage.pm has a syntax error, this will now catch it
during build

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 10:27:51 +02:00
c86ba76db7 LVMPlugin: factor out the lv creation
we will want to create lvs manually for ceph nautilus db/wal devices

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-06-04 10:26:46 +02:00
0180fa427f followup: get_disks: use own variable for frequent access
Less reading and the own name for the variable should helps to grasp
more quickly what it should contain

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-31 12:04:42 +02:00
248f43f58b follouwp: get_ceph_volume_infos: code and comment cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-31 12:04:13 +02:00
19dcd1adcb Diskmanage: detect osds/journals/etc. created with ceph-volume
ceph-volume creates osds/journal/etc. on LVM instead of partitions,
so to detect them, we have to parse the lv_tags of the LVs and
match them with the underlying device

also add tests for this detection

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-05-31 11:41:33 +02:00
cd814c0453 allow disk with LVM for journal devices
With this, users can select disks with LVM on it for journal/db devices,
which will be necessary for ceph nautilus, since there
journals/db/wal will be put on an LV

of course when creating an osd, we have to detect if that
is ok (probably based on the vg name on it)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-05-31 11:41:33 +02:00
08335d55c9 buildsys: no need to include arch detection for arch-independent package
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-21 22:24:42 +02:00
f3cb30b992 buildsys: switch upload dist over to buster
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-21 20:57:05 +02:00
5518d36142 bump version to 6.0-0+1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-21 20:54:49 +02:00