Commit Graph

1466 Commits

Author SHA1 Message Date
a1e09e496e iscsi: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-04 12:02:47 +02:00
9177cc2eda clone image: specify base format option with qemu-img
and avoid a warning. It is deprecated to auto-detect the format of the base
volume. See commit d9f059aa6cfccefaffa3532556e966df4a99ece2 in qemu for more
information.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-05-03 13:07:02 +02:00
dc348496d0 bump version to 6.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 16:18:47 +02:00
fef9987d4a d/control: depend on proxmox-backup-file-restore
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 15:34:27 +02:00
4708d32180 d/control: bump versioned dependency for libpve-common-perl
for the new file-restore helpers in PBSClient which we use in the API
here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 15:15:56 +02:00
c1ec1acbde file-restore: pass in volume ID or name
instead of just the snapshot for consistency with other API endpoints,
and possible future extension to VMA backups (where 'snapshot' would be
a rather strange terminology).

add some additional checks (pbs storage type, backup volume type),
completion and magic (allow passing in either a full volume ID with
correct storage, or just the volume name, or just the snapshot for
easier API/CLI usage/convenience).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-23 14:09:54 +02:00
82f764e119 file-restore: return perl-y booleans
like we do in most of our API.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-23 14:09:54 +02:00
f1a3ce3b17 add FileRestore API for PBS
Includes list and restore calls.

Requires VM.Backup and Datastore.Audit permissions, for the accessed
VM/CT and containing datastore respectively.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-23 14:09:48 +02:00
6035a5dfb1 api: fix typo in error message
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-23 12:29:36 +02:00
c56f7a71af pbs: allow setting up a master key
similar to the existing encryption key handling, but without
auto-generation since we only have the public part here.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-22 21:56:31 +02:00
3c93115570 rbd: fix typo in error message
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-20 18:21:31 +02:00
ceb7b1ed09 diskmanage: get_partnum: fix check
Not replacing it with return, because the current behavior is dying:
    Can't "next" outside a loop block
and the single existing caller in pve-manager's API2/Ceph/OSD.pm does not check
the return value.

Also check for $st, which can be undefined in case a non-existing path was
provided. This also led to dying previously:
    Can't call method "mode" on an undefined value

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-20 18:13:18 +02:00
415dc3985d diskmanage: improve setting usage for whole disk with include-partitions
in case a disk with partitions also has an fstype set, which happens for our ZFS
boot disks. Do not change the behavior without include-partitons, as we
prefer(red) to be more specific than simply 'partitions' then.

Reported in the enterprise support channel.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-16 12:52:08 +02:00
1ebd925dcf import: allow import from UNIX socket
this allows forwarding over websockets without requiring a (free) port.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-16 12:23:43 +02:00
9e47ad47ec bump version to 6.3-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 16:14:36 +02:00
0d8ed0a313 test: rbd: improve usage output
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:43:42 +02:00
23ba4ac2c6 test: rbd: typo fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:27:33 +02:00
33191d3266 test: rbd: indendation cleanup, allow -d short-opt for debug
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:27:26 +02:00
0d7c9ca188 test: rbd: actually use debug helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 15:27:23 +02:00
bef7920d1e volume export/import: allow uppercase letters
Bug reported in the community forum[0].

Currently, it's possible to break replication by:
1. have an existing snapshot whose name contains an uppercase letter
2. set up a replication job and run it
3. rollback to the existing snapshot
4. replicate again -> fails

The failure occurs, because after step 3, the most recent common snapshot is the
previously existing one and currently no uppercase letters are allowed for
export/import.

The pve-snapshot-name option uses the CONFIGID_RE
    qr/[a-z][a-z0-9_-]+/i
so it cannot be used here, because it would not allow for e.g. '__migrate__'.
Simply allow uppercase letters, to be backwards compatible and allow all
possible pve-snapshot-name values.

There is still an issue if there also was state volume, but that's a different
bug[1].

[0]: https://forum.proxmox.com/threads/solved-migration-error-base-value-does-not-match-the-regex-pattern.85946/
[1]: https://bugzilla.proxmox.com/show_bug.cgi?id=3111

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-12 14:52:29 +02:00
9407a7a338 tests: rbd ns: exit cleanly on help, and allow short-opt -h
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 14:46:12 +02:00
876d268298 tests: rbd ns: avoid underscore in CLI param name
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 14:42:38 +02:00
8c858f7eeb fix #3345: zfs: restore container volume to ZFS with size 0
A restore to ZFS for a container which has a volume (rootfs / mount
point) of size 0 failed because the refquota property does not accept
'0k' but wants 'none' in that situation.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 14:37:50 +02:00
c27fe64810 rbd: make volume param for get_rbd_path to allow further use
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 14:19:48 +02:00
ed7ea5a352 rbd: list images: early return to avoid indentation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 13:51:15 +02:00
a3cad0b50d rbd: list images: sort by keys when pushing on result array
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 13:49:56 +02:00
6d0d0a977d rbd: indentation and whitespace cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 13:48:27 +02:00
22265bd990 rbd: get kernel device sub returns a path, not a name
also transform to private sub instead of local variable closure.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 13:45:21 +02:00
72bbd8a6f7 rbd: consistent closure call style
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 13:43:33 +02:00
92a7826f88 rbd: build cmd: allow "falsy" namespace value also here
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 13:43:33 +02:00
4cf696f66e rbd: use private sub for get_rbd_path
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 13:43:33 +02:00
9f3d6f3bb9 rbd: add integration test for namespace handling
This test is intended to be run on a hyperconverged PVE cluster to test
the most common operations of VMs using a namespaced Ceph RBD pool.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-04-09 12:56:21 +02:00
e9bc993138 rbd: fix #3286 add namespace support
This patch introduces support for Cephs RBD namespaces.

A new storage config parameter 'namespace' defines the namespace to be
used for the RBD storage.

The namespace must already exist in the Ceph cluster as it is not
automatically created.

The main intention is to use this for external Ceph clusters. With
namespaces, each PVE cluster can get its own namespace and will not
conflict with other PVE clusters.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-04-09 12:56:21 +02:00
aeb007cb76 rbd: centralize rbd path concatenation
The <pool>/<image> paths are needed in quite a lot of places. Having one
single place where they are created helps to reduce duplicate code and
makes it easier to introduce new features.

The 'add_pool_to_disk' sub was already doing that but the name was not
really fitting. This commit renames it to the more general
'get_rbd_path' and changes the second parameter to the more widely used
$volume instead of $disk.

Furthermore, all occurences where "$pool/$volume" has been concatenated
have been replaced with a call to get_rbd_path.

Plus some minor code style cleanups for long function calls that were
touched.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-04-09 12:56:21 +02:00
cba2b7c1d4 prune backups: improve internal errors messages slightly
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 12:49:55 +02:00
4b702ac361 plugin: get_subdir_files: add comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 12:46:46 +02:00
68ce0b31e8 prune backups: make vmid filtering more robust
by relying on archive_info's vmid first. archive_info is already used to
determine if it's a standard name, and in that case the vmid is certainly set.

Also add asserts to make sure we got what we expected.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-09 12:41:34 +02:00
edb92f707a plugin: subdir files: backup: don't match for vmid against the full path
Only match against the file name to avoid false positives with
directory names containing "-$vmid-".

Found while trying to debug/reproduce a forum thread[0], but the path
there should not be affected by this...

[0]: https://forum.proxmox.com/threads/vzdump-removing-too-many-backups.87072/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-09 12:41:10 +02:00
c241c2159d bump version to 6.3-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-01 15:04:16 +02:00
4b84ad5e25 Optionally allow blockdev in abs_filesystem_path
This is required to import from LVM storages, for example

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-03-31 11:00:25 +02:00
342a56805c fix #3348: NFS: select correct transport to check for service
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-31 10:22:52 +02:00
dfa374d320 fix #3363: avoid undef-warning for PBS crypt-mode
it is optional after all, and missing (/None) for files stored in the
snapshot dir but not referenced in the manifest for whatever reason.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-03-31 10:22:52 +02:00
2c5246e1ea vdisk list: allow specifying content type
and only scan storages that support it if specified.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-31 10:22:52 +02:00
26d7314be0 Revert "vdisk list: only collect images from storages with an appropriate content type"
This reverts commit a44c18925d and adds a reminder
comment.

The mentioned commit is actually a backwards-incompatible change that leads to
slightly different behavior when migrating a VM with volumes on a misconfigured
storage. For example, unreferenced volumes on a misconfigured storage won't be
picked up, even though they were before. And for referenced volumes on a
misconfigured storage, the disk size would not be updated on migration anymore.

We should wait until the next major release for this change and then also
re-evaluate the migration behavior with misconfigured disks.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-31 10:22:52 +02:00
2bce96c513 fix #3354: support notes on ceph backups
use DirPlugin's get/update_volume_notes implementation (which all the
other supported file systems use)

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-03-31 10:22:52 +02:00
061c7e5e3d tests: zfs: complain when a first sub-test dies
Previously, $count was not increased and no test failure was reported.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-15 13:36:53 +01:00
a44c18925d vdisk list: only collect images from storages with an appropriate content type
Only these storages are activated in the first place, and it's bad behavior to
list images when no appropriate content type is not set.

For example, on VM destruction, this avoids unreferenced images to be deleted
from a storage with only 'backup' content type set, which is supposedly what
happened in this[0] forum thread.

(Some) callers expect all keys to be present and valid array references in the
result, so initialization is needed.

Now, the enabled check is already done by the preceding code for every element
that is iterated over, and thus isn't needed in the main loop anymore.

[0]: https://forum.proxmox.com/threads/erasing-all-vm-disks-after-a-failed-vm-migration-task.85068

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-15 13:36:53 +01:00
1d6c548889 api: disk list: allow if an audit permission for the node is present
as that seems to be the more natural permission path for listing a nodes local
disks. For backwards compatibility, the old permission check has to be kept
(relevant with propagate=0).

This API call was originally part of the Ceph API and got copied here later,
which might explain the current permission check.

In the UI, the Disk panel is visible with a node audit permission, but the API
call itself failed without the '/' audit permission.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-03-11 14:16:21 +01:00
6c460330bb d/control: record pve-qemu-kvm | qemu-utils build depenency for tests
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-02 17:20:57 +01:00
977b80c8ab disks: zfs: scan is only returned optionally
the line is not present if a zpool has never been scrubbed before
(e.g. if it's freshly setup)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-02-26 09:10:08 +01:00