Commit Graph

91 Commits

Author SHA1 Message Date
4de6002558 diskmanage: add mounted_paths
returns a list of mounted paths with the backing devices

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2022-09-13 10:04:48 +02:00
2949acd638 diskmanage: only set mounted property for mounted devices
instead of setting an empty string for not mounted devices

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-15 10:49:22 +02:00
76c98d2353 api2: disks: add mounted boolean field
... and remove '(mounted)' from usage string

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-06-15 10:17:10 +02:00
26082b7daf diskmanage: add helper for udev workaround
to avoid duplication. Current callers pass along at least one device,
but anticipate future callers that might call with the empty list. Do
nothing in that case, rather than triggering everything.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-10 12:35:25 +01:00
a2c34371e6 partially fix #2285: api: disks: allow partitions for creation paths
The calls for directory and ZFS need slight adaptations. Except for
those, the only thing that needs to be done is support partitions in
the disk_is_used helper.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-10-07 14:39:26 +02:00
cc884f73d8 api: disks: initgpt: explicitly abort for partitions
In preparation to extend disk_is_used to support partitions. Without
this new check, initgpt would also allow partitions once disk_is_used
supports partitions, which is not desirable.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-10-07 14:39:26 +02:00
ff91cfae35 diskmanage: don't set usage for unused partitions
The disk type is already 'partition' so there's no additional
information here. And it would need to serve as a code-word for
unused partitions. The cleaner approach is to not set the usage.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-10-07 14:39:26 +02:00
bd46e59b5d diskmanage: wipe blockdev: also change partition type
when called with a partition. Since get_disks uses the partition type
(among other things) to detect LVM and ZFS volumes, such volumes would
still be seen as in-use after wiping. Thus, also change the partition
type and simply use 0x83 "Linux filesystem".

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-10-07 14:39:26 +02:00
e8df8fb179 diskmanage: add change_parttype and is_partition helpers
For change_parttype, only GPT-partitioned disks are supported, as I
didn't see an option for sgdisk to make it also work with
MBR-partitioned disks. And while sfdisk could be used instead (or
additionally) it would be a new dependency, and AFAICS require some
conversion of partition type GUIDs to MBR types on our part.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-10-07 14:39:26 +02:00
a64aedd3eb diskmanage: allow passing partitions to get_disks
Requires that the $include_partitions parameter is set too, which:
1. Makes sense, because the partition won't be included in the result
   otherwise.
2. Ensures backwards compatibility for existing callers that don't
   use $include_partitions. No existing callers use both $disks and
   $include_partitions at the same time, so nothing learns to
   "support" partitions by accident.

Moving the strip_dev helper to the top, so it can be used everywhere.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-30 18:11:00 +02:00
0f4533476d diskmanage: allow partitions for get_udev_info
both existing callers only call this with non-partitions currently, so
the change should be backwards compatible.

In preparation to enable ZFS creation on top of partitions (where the
udev info is used to get the stable by-id path of a device).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-09-30 18:10:56 +02:00
ffc31266da tree-wide: fix typos with codespell
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 08:28:48 +02:00
f7a95153d6 diskmanage: fix determining array length
$#* is the last index, not the length.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-15 10:10:33 +02:00
d938178298 disks: fixup join usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-02 14:19:53 +02:00
839afff896 disks: wipe blockdev: pass all child partitions to wipefs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-02 13:13:26 +02:00
fa6d05ab24 disks: wipe blockdev: improve variable locality/readability
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-02 13:12:57 +02:00
70dc70984a disks: factor out stripping of /dev and cleanup vicinity
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-02 13:10:10 +02:00
cb057e21c5 diskmanage: add has_holder method
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-02 11:56:51 +02:00
3bf7f8891b diskmanage: add is_mounted method
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-02 11:56:51 +02:00
7e14102a4b diskmanage: factor out mounted_blockdevs helper
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-02 11:56:51 +02:00
262ad7a92e diskmanage: add wipe_blockdev method
based on the wipe_disks method from pve-manager's Ceph/Tools.pm with the
following main differences:
    * use wipefs to wipe labels first (to avoid sgdisk complaining about the
      backed up GPT structure on a subsequent GPT initialization)
    * only take one device as an argument
    * do not use an absolute path for 'dd'
    * die if one of the command fails

The wipefs command makes checks and complains about e.g. mounted or active
devices. One could supply --force to wipefs, but in many such situations it
does not work as expected, because the device would still be detected as in-use
afterwards, and further manaual steps would be needed.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-02 11:56:51 +02:00
ceb7b1ed09 diskmanage: get_partnum: fix check
Not replacing it with return, because the current behavior is dying:
    Can't "next" outside a loop block
and the single existing caller in pve-manager's API2/Ceph/OSD.pm does not check
the return value.

Also check for $st, which can be undefined in case a non-existing path was
provided. This also led to dying previously:
    Can't call method "mode" on an undefined value

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-20 18:13:18 +02:00
415dc3985d diskmanage: improve setting usage for whole disk with include-partitions
in case a disk with partitions also has an fstype set, which happens for our ZFS
boot disks. Do not change the behavior without include-partitons, as we
prefer(red) to be more specific than simply 'partitions' then.

Reported in the enterprise support channel.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-16 12:52:08 +02:00
525b4a6ebe Diskmanage: replace check for zpool binary with a function and mock it
so the test still works when it's not installed.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-19 15:29:04 +01:00
31ed94ccf8 Diskmanage: also set type for partitions
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-09 11:56:48 +01:00
2949c537d9 Diskmanage: also include partitions with get_disks if flag is set
and have a parent key for partitions, to be able to see the associated disk in
the result without having to rely on naming heuristics (just adding a number at
the end doesn't work for NVMes).

The disk's usage will not be based on the partitions usage if the flag is set,
but will simply be 'partitions'.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
6a1919b137 Diskmanage: save OSD information for individual partitions
in preparation to including partitions for get_disks()

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
41f93eceb2 Diskmanage: introduce ceph info helper
so it can be re-used for partitions.

Also changes the regular expression in get_ceph_volume_info to match the full
device/partition name the LV is on. Not only is this needed for partitions,
especially if there's multiple partitions with an OSD, but it also fixes
handling NVMe devices with an OSD as a side effect. Previuosly those were not
detected here, because of the digits in the name, e.g. /dev/nvme0n1

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
d3857eebb5 Diskmanage: also detect BIOS boot, EFI and ZFS reserved type partitions
as they are relevant to most PVE setups.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
01aa7d75fd Diskmanage: introduce usage helper
Note that this is a slight behavior change, because now the first
partition's usage which is not simply 'partition' will become the disk's
usage. Previously, if any partition was 'mounted', it would become the disk's
usage, then 'LVM', 'ZFS', etc.

A partitions usage defaults to 'partition' if nothing more specific can be
found, and is never treated as unused for now.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
89c27ea8ab Diskmanage: collect partitions in hash
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
40be5c5cab Diskmanage: introduce get_sysdir_size helper
to be used for partitions as well.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
59c03cd970 Diskmanage: also check for filesystem type when determining usage
Like this, a non-ZFS filesystem living on a whole disk will also be detected
when it is not mounted.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
b6bbc2ab28 Diskmanage: refactor and rename get_parttype_info
in preparation to also query the file system type from lsblk. Note that the
result now also includes devices without a parttype, so a definedness check in
get_devices_by_partuuid is needed. This will be useful when the whole device
contains a filesystem.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
0cca5356bb Diskmanage: replace closure with direct hash access
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
d3a5e30963 drop absolute udevadm path
the compat symlink from bin to sbin has been dropped with bullseye, and
we rely on PATH begin set properly in our daemons/CLI tools anyway..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 18:27:14 +01:00
2c048efdaf Diskmanage: extend wearout detection for SAS disk
for some controllers/disks there the line is
Percentage used endurance indicator: x%

so extend the regex for that possibilty.
We even had a test-case for SAS but did not notice we could extract
that info from there...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-15 14:42:16 +01:00
5045e0b77a perlcritic: don't modify $_ in list functions, use for
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-23 15:27:14 +01:00
1207620cf0 perlcritic: avoid conditional variable declaration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-23 15:24:02 +01:00
dbad606d57 Diskmanage: Use S.M.A.R.T. attributes for SSDs wearout lookup
This replaces a locally maintained hardware map in
get_wear_leveling_info() by commonly used register names of
smartmontool. Smartmontool maintains a labeled register database that
contains a majority of drives (including versions). The current lookup
produces false estimates, this approach hopefully provides more reliable
data.

Signed-off-by: Jan-Jonas Sämann <sprinterfreak@binary-kitchen.de>
2020-10-30 15:25:11 +01:00
4731eb1118 disk management: set more specific type for nvme
some users are confused, and it's nicer to have the more specific
type presented here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-09 18:06:38 +02:00
d5c80a5bd5 code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-09 18:06:07 +02:00
0f0d99a3e5 fix #2777 create zpools with stable dev paths
when compiling the disk list add a property with a stable
/dev/disk/by-id/ path for a block device when available.

This is needed to create zpools with the stable by-id links

The /dev/disk/by-id/ directory can contain multiple links to the same device
(e.g. when it's used as a LVM PV, or one for the wwn/nvme-eui in addition
to the one with vendor and serial). We take the first one which matches
the bus where the disk is attached. For nvme disks we exclude the one
containing the nvme-eui.

The patch assumes that not all disks need to have such a link (e.g.
virtio-block devices as we pass them to guests).

Additionally the tests were adapted to run successfully.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-06-06 19:32:33 +02:00
92ae59df9e storage: replace build-in stat occurrences
with File::stat::stat to minimize variable declarations. And allow to
mock this method in tests instead of the perl build-in stat.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
ce73713ef7 fix nvme wearout parsing
the '.*' was greedy, also consuming all but one digits of the real percentage

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>

switch to \s* instead of .*?, to prevent mis-interpreting potential
strings like '< 50%' or '0-50%'

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-15 12:54:10 +02:00
481f6177a5 grammar fix: s/does not exists/does not exist/g
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-13 12:11:26 +01:00
bfb3d42daf Diskmanage: add flag for encrypted osds
we can only do this here, since the ceph cluster is not aware of
osd encryption, only the local node is (via ceph-volume and lv tags)

this way, we are able to show an 'encrypted' flag in the disk gui at least

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:38:24 +02:00
8cd6d7e8e3 Diskmanage: change parttype uuid detection
previously ceph included a udev rule to populate
/dev/disk/by-parttypeuuid/

but not anymore, so we now use 'lsblk --json -o path,parttype' to
get a mapping between parttype uuid and partition

fix the test by simulating empty lsblk output

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-12 15:37:57 +02:00
79f4a7bfd3 fix #2266: Diskmanage: get correct osd id
The osdid can consist of multiple digits, cope with that and add more
regression tests for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-08 19:04:08 +02:00
35ccd3d004 fix missing osd info for osd 0
0 is falsy, we have to check for definedness
also adapt the tests so we test for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-07-03 08:59:47 +02:00