Commit Graph

1395 Commits

Author SHA1 Message Date
59c03cd970 Diskmanage: also check for filesystem type when determining usage
Like this, a non-ZFS filesystem living on a whole disk will also be detected
when it is not mounted.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
b6bbc2ab28 Diskmanage: refactor and rename get_parttype_info
in preparation to also query the file system type from lsblk. Note that the
result now also includes devices without a parttype, so a definedness check in
get_devices_by_partuuid is needed. This will be useful when the whole device
contains a filesystem.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
0cca5356bb Diskmanage: replace closure with direct hash access
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
b2843044ba Disks: return correct journal disk candidates
Previously any GPT initialized disk without an osdid (i.e. equal to -1) would
be included in the list of journal disk candidates, for example a ZFS disk. But
the OSD creation API call will fail for those. To fix it, re-use the condition
from the corresponding check in that API call (in PVE/API2/Ceph/OSD.pm).
Now, included disks are unused disks, those with usage 'partitions' and GPT, and
those with usage 'LVM'.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 13:52:20 +01:00
acc9c3acbc api: storage/config: use extract_sensitive_params from tools
we have a more general version there

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 17:31:19 +01:00
68f1fc2783 mark PBS storages as shared
Like this, the property will get added when parsing the storage configuration
and PBS storages will correctly show up as shared storages in API results.

AFAICT the only affected PBS operation is free_image via vdisk_free, which will
now be protected by a cluster-wide lock, and that shouldn't hurt.

Another issue this fixes, which is the reason this patch exists, was reported
in the forum[0]. The free space from PBS storages was counted once for each node
that had access to the storage.

[0]: https://forum.proxmox.com/threads/pve-6-3-the-storage-size-was-displayed-incorrectly.83136/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-01-28 16:43:20 +01:00
d854a71803 lvm: Fix #3159: Show RAID LVs as storage content
LVM RAID logical volumes (including mirrors) can be valid disk images, so they
should show up in storage content listings (for example pvesm list).

Including LV types is safer than excluding, especially because of possible
additional types in the future.

Co-developed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-01-26 19:19:02 +01:00
89b9ac961a fix: check connection for nfs v4 only server
the check_connection is done by querying the exports of the nfs server
in question. With nfs v4 those exports aren't listed anymore since nfs
v4 employs a pseudo-filesystem starting from root (/).

rpcinfo allows to query the existence of an nfs v4 service.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2021-01-26 19:15:17 +01:00
d7fedf266a bump version to 6.3-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-26 18:37:38 +01:00
26a8f21ab2 add workaround for zfs rollback bug
as described in the zfs bug https://github.com/openzfs/zfs/issues/10931
the kernel keeps around cached data from mmaps after a rollback, thus
having invalid data in files that were allegedly rolled back

to workaround this (until a real fix comes along), we unmount the subvol,
invalidating the kernel cache anyway

the dataset gets mounted on the next 'activate_volume' again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 18:32:47 +01:00
d3a5e30963 drop absolute udevadm path
the compat symlink from bin to sbin has been dropped with bullseye, and
we rely on PATH begin set properly in our daemons/CLI tools anyway..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 18:27:14 +01:00
a7e0184190 debian/control: use https in homepage link
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-26 14:04:15 +01:00
2c048efdaf Diskmanage: extend wearout detection for SAS disk
for some controllers/disks there the line is
Percentage used endurance indicator: x%

so extend the regex for that possibilty.
We even had a test-case for SAS but did not notice we could extract
that info from there...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-15 14:42:16 +01:00
a66db4ae55 tests: add ssd sas disk
copied from test 'sas' with rotational set to 0
this has then the type 'ssd', rpm: 0, and health: 'OK'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-15 14:42:16 +01:00
189e67ffa6 fix #3199: by fixing usage of strftime
In a very early version I wanted to parse the date from the backup
name, and when switching to using the ctime and localtime() instead,
I forgot to update the usage of strftime.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-15 14:40:45 +01:00
32dbc619a5 drbd: comment that the builtin plugin is depreacated
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-15 12:58:20 +01:00
c6e632efb6 d/control: update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-14 21:58:24 +01:00
175d73fa75 d/compat: update to 10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-14 21:53:13 +01:00
b59f5e0eed bump version to 6.3-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-14 16:15:07 +01:00
10dfeb9ef0 prune mark: correctly keep track of already included backups
This needs to happen in a separate loop, because some time intervals are not
subsets of others, i.e. weeks and months. Previously, with a daily backup
schedule, having:
* a backup on Sun, 06 Dec 2020 kept by keep-daily
* a backup on Sun, 29 Nov 2020 kept by keep-weekly
would lead to the backup on Mon, 30 Nov 2020 to be selected for keep-monthly,
because the iteration did not yet reach the backup on Sun, 29 Nov 2020 that
would mark November as being covered.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-14 16:11:29 +01:00
44fdfb2af6 nfs and cifs: implement backup notes helper
reuse the one from DirPlugin by directing the call to it, but with
the actual $class. This should stay stable, as we provide an ABI and
try to always use $class->helpers.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-07 16:13:09 +01:00
f244e2aa7f api: content/backup: handle deletion of notes
Previous to this we did not called the plugins update_volume_notes at
all in the case where a user delted the textarea, which results to
passing a falsy value ('').

Also adapt the currently sole implementation to delete the notes field
in the undef or '' value case. This can be done safely, as we default
to returning an empty string if no notes file exists.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-07 16:10:09 +01:00
20471dfd95 dir plugin: code cleanup
mostly re-ordering to improve statement grouping and avoiding the
need for an intermediate variable

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-07 16:09:26 +01:00
a8e00d1a5d bump version to 6.3-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-03 17:24:59 +01:00
b1f50fb5cc d/control: bump versioned dependency for libpve-common-perl
so the new get_repository helper for PBS is available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-03 16:55:28 +01:00
53003cb5ea PBSPlugin: use get_repository from PVE::PBSClient
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 16:53:53 +01:00
ab90c3b1f1 pbs: fix token auth with PVE::APIClient
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-03 16:53:43 +01:00
405b247168 api: scan: note that USB is depreacated
It now got moved in /nodes/<node>/hardware/usb as envisioned[0], this
allows to sunset the usb scan API endpoint here and drop it with 7.0

[0]: https://lists.proxmox.com/pipermail/pve-devel/2018-November/034694.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-02 11:31:06 +01:00
a57101434b bump version to 6.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:27:54 +01:00
39fc0c77e2 api/cli: add pbs scan endpoint and command
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
11a942a18a pvesm: also map the password param for new style cifs scan command
not only for the old deprecated alias

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
b1ccf4f357 api: scan: move over index and usb scan from manager
Add the missing pieces allowing pve-manager to just point the
/nodes/<node>/scan api directory at this module, dropping it's
duplicated copy.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
e65abdb458 api: scan cifs: port over NT_STATUS filter from pve-manager
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
2892b6739d factor out scan CLI definition to real API module
we have a 1:1 copy of that code in pve-manager's PVE::API2::Scan,
which we can avoid by using a common module form pvesm CLI and the
API.

This is the first basic step of dropping the code duplication in
pve-manager.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
2cd10f58be pbs: activate storage: fully validate if storage config works
improves UX of on_update and on_add hooks *a lot*.

This is a bit more expensive than the TCP ping, or even just an
unauthenticated ping, but not as bad as a full datastore status - as
this only reads the datastore config file (which is normally in page
cache anyway).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
8b62ac6a0c pbs: add scan datastore helper
for use in both, the scan API and the on_add/on_update hooks

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
2f9eb6dc4c pbs: reuse pve apiclient for api connect helper
it is flexible enough to easily do so, and should do well until we
actually have cheap native bindings (e.g., through wolfgangs rust
permlod magic).

Make it a private helper, we do *not* want to expose it directly for
now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
f3ccd0ef3d plugin: hooks: add explicit returns
to avoid returning something unexpected. Finish what
afeda18256 already started for all the other
plugins. At least for ZFS's on_add_hook this is necessary (adding a ZFS storage
currently fails as reported here [0]), but it cannot hurt
in the other places either as the only hooks we expect to return something
currently are PBS's on_add_hook and on_update_hook.

[0]: https://forum.proxmox.com/threads/gui-add-zfs-storage-verification-failed-400-config-type-check-object-failed.79734/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-27 10:45:42 +01:00
74dcca3a48 nfs: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-27 10:45:06 +01:00
39591d92ec bump version to 6.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 23:20:58 +01:00
878fe0177e api: content: pass encrypted status for PBS backups
Prefer the fingerprint, fallback to checking the files crypt-mode.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 23:18:05 +01:00
3cc2eb738f pbs add/update: save fingerprint in storage config
fallback to the old truthy "1" if not available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:09:40 +01:00
d2c47b3837 pbs add/update: do basic key value validation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:09:15 +01:00
478609d3bc pbs: autogen key: rename old one if existing
it could be debated do have some security implications and that
deletion is safer, but key deletion is a pretty hairy thing.

Should be documented, and people just should use delete instead of
autogen if they want to "destroy" a key.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:05:23 +01:00
7e6a7a7f39 bump version to 6.2-12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 16:06:03 +01:00
45e93e6dda Storage/PBSPlugin: implement get/update_volume_notes for pbs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 13:42:13 +01:00
e9991d2694 Storage/Plugin: add get/update_volume_comment and implement for dir
and add the appropriate api call to set and get the comment
we need to bump APIVER for this and can bump APIAGE, since
we only use it at this new call that can work with the default
implementation

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
3369651885 api2/storage/content: change to volume_size_info and add return properties
'file_size_info' only works for directory based storages, while
'volume_size_info' should work for all

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
6fef456c8d rename comment to notes
so that we are more consistent with pbs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
093d5ad385 bump version to 6.2-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-23 19:37:00 +01:00