Commit Graph

613 Commits

Author SHA1 Message Date
a0933d7e16 prune-backups CLI: use keep-options directly
Makes the interface cleaner; e.g. --keep-daily=2 instead of
--prune-backups=keep-daily=2

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-09-03 18:20:06 +02:00
4ef17e1fe0 pbs: check if encryption key could be deleted
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:06:09 +02:00
18cf6c9f27 config: add hint for set encryption key, die on unlink error
as else the API cannot easily know if this is set, it cannot check
with -f as the key is in a restricted area and we do not want that a
GET runs as protected.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:05:25 +02:00
8f26b3910d Add prune_backups to storage API
Implement it for generic storages supporting backups
(i.e. directory-based storages) and add a wrapper for PBS.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
3353698f45 Introduce prune-backups property for directory-based storages
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
02cc5e1033 quick fixup for prune command in vzdump
This is a hack and we should get rid of `run_client_cmd` and
`run_raw_client_cmd` as an API entry!

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 14:55:07 +02:00
bb0a0f9680 pbs: activate_storage: query status to ensure we can connect
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:54:13 +02:00
1aeb322bc3 pbs: allow to autogen an encryption key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:53:51 +02:00
ce2e273359 pbs: rename encryption_key param to encryption-key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:53:27 +02:00
72385de9e2 refactor sensitive parameter handling
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 11:49:02 +02:00
76bb5febc7 pbs: encryption support, split "raw client command" API
(And deprecate it...)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 11:49:02 +02:00
0f2549edd4 cifs: use more general name for password description
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:48:52 +02:00
245b651789 fix #2460: use dm-uuid-mpath- as fallback for multipath
With Debian Buster it looks like the 'scsi-' method is no longer
reliable. In addition to that, which is also used for non-multipath
systems, add the 'dm-uuid-mpath-' method as fallback. This is also used
by openstack (see os-brick
39b201160b/os_brick/initiator/linuxscsi.py (L400))

Also sort the output of readdir so 'scsi-' is always after
'dm-uuid-mpath-' so the output of pvesm list does not change for systems
that worked before.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-07-07 16:58:25 +02:00
e6d1edcb93 pbs: run_raw_client_cmd: fix typo, and drop repetive info
It would be s/bps/pbs/ but as we already have "proxmox-backup-client"
included in the log through the executable name, so it should be
clear that this is a PBS command - so drop that part entirely.

Now using:
> run: /usr/bin/proxmox-backup-client ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-07 10:15:59 +02:00
8602fd5646 PBSPlugin: Add print_volid helper
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-30 13:59:23 +02:00
e9667b3418 base plugin: fall back to empty hash to avoid checking archive info definedness
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-06 19:40:10 +02:00
fb821c1828 Expand archive_info to include ctime, vmid and is_std_name
where 'is_std_name' shows whether the backup name uses the standard naming
schema and most likely was created by our tools.

Also adds a '^' to the existing filename matching regex, which
should be fine since basename() is used beforehand.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-06 19:38:02 +02:00
ddf7fdaa82 PBSPlugin: list_volumes: filter by vmid if specified
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-06 19:38:02 +02:00
8b4c2a7ec6 PBSPlugin.pm: avoid ascii-art in proxmox-backup-client output 2020-06-03 11:07:01 +02:00
3d10acf89e Fix #2737: Can't call method "mode"
on an undefined value at /usr/share/perl5/PVE/Storage/Plugin.pm line 928

This error message crops up when a file is deleted after getting the
file list and before the loop passed the file entry.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-05-15 18:12:01 +02:00
014d36dbbb Fix: #2124 storage: add zstd support
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
277cafc0ff backup: compact regex for backup file filter
the more compact form of the regex should allow easier addition of new
file extensions.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
40c795e7df Fix: backup: ctime was from stat not file name
The vzdump file was passed with the full path to the regex. That regex
captures the time from the file name, to calculate the epoch.

As the regex didn't match, the ctime from stat was taken instead. This
resulted in the ctime shown when the file was changed, not when the
backup was made.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
c48801b52a test: parse_volname
Test to reduce the potential for accidental breakage on regex changes.
And to make sure that all vtype_subdirs are parsed.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
92ae59df9e storage: replace build-in stat occurrences
with File::stat::stat to minimize variable declarations. And allow to
mock this method in tests instead of the perl build-in stat.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
7435dc9071 s/ceph_version/local_ceph_version/ for clarity
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-25 11:37:26 +02:00
d4c31eff96 d/control: bump ceph dependency to 12.2
A newer than the Luminous version is shipped with buster, and our
ceph repos are on Nautilus (14.2) in PVE 6.

Allows to drop a check for really old ceph versions (< 10, so
Infernalis and older).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-25 11:27:38 +02:00
81c5c736ca followup: only parse version if required, fix whitespace error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-25 11:18:03 +02:00
e54c3e3347 Fix #2705: cephfs: mount fails with bad option
dmesg: libceph: bad option at 'conf=/etc/pve/ceph.conf'

After the upgrade to PVE 6 with Ceph Luminous, the mount.ceph helper
doesn't understand the conf= option yet. And the CephFS mount with the
kernel client fails. After upgrading to Ceph Nautilus the option exists
in the mount.ceph helper.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-25 11:15:23 +02:00
e05113fbe5 ZFS: use -p flag where possible
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-04-09 10:20:06 +02:00
3881e68025 ZFS: use -p flag and remove zfs_parse_size
ZFS supports the -p flag in the list command since a few years now.
Let us use the real byte values and avoid the error prone calculation
from human readable numbers that can lead to incorrect numbers if the
reported human readable value is a rounded number.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-04-09 10:20:06 +02:00
d99de0f898 ZFSPoolPlugin: fix #2662 get volume size correctly
Getting the volume sizes as byte values instead of converted to human
readable units helps to avoid rounding errors in the further processing
if the volume size is more on the odd side.

The `zfs list` command supports the -p(arseable) flag since a few years
now.
When returning the size in bytes there is no  calculation performed and
thus we need to explicitly cast the size to an integer before returning
it.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-04-09 10:19:59 +02:00
a97d3ee49f Introduce allow_rename parameter for pvesm import and storage_migrate
and also return the ID of the allocated volume. This option
allows plugins to choose a new name if there is a collision.

In storage_migrate, the API version of the receiving side is checked.

In Storage.pm's volume_import, when a plugin returns 'undef',
it can be assumed that the import with the requested volid was
successful (it should've died otherwise) and so volid is returned.
This is done for backwards compatibility with foreign plugins.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-09 09:41:01 +02:00
b9364dc683 Fix 2647: Add snippet content type for Gluster
Our wiki mentions snippets as supported content type for GlusterFS storages [0]
and all other directory based storages have it enabled already [1]

[0] https://pve.proxmox.com/wiki/Storage:_GlusterFS
[1] https://git.proxmox.com/?p=pve-storage.git;a=commit;h=d1eb35ea74cf27713625ab7e7c3767a8254a4aee

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2020-04-08 07:53:17 +02:00
3587acc80a fix #2474: always show iscsi content
Instead of relying on list_volumes of Plugin.pm (which filters by
the content types set in the config), use our own to always
show the luns of an iscsi.

This makes sense here, since we need it to show the luns when using
it as base storage for LVM (where we have content type 'none' set).

It does not interfere with the rest of the GUI, since on e.g. disk
creation, we already filter the storages in the dropdown by content
type, iow. an iscsi storage used this way still does not show up
when trying to create a disk.

This also shows the luns now in the 'Content' tab, but this is also
OK, since the user cannot actually do anything there with the luns.
(Besides looking at them)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-04-07 18:09:50 +02:00
e6f4eed435 Allow passing options to volume_has_feature
With the option valid_target_formats it's possible
to let the caller specify possible formats for the target
of an operation.
[0]: If the option is not set, assume that every format is valid.

In most cases the format of the the target and the format
of the source will agree (and therefore assumption [0] is
not actually assuming very much and ensures backwards
compatability). But when cloning a volume on a storage
using Plugin.pm's implementation (e.g. directory based
storages), the result is always a qcow2 image.

When cloning containers, the new option can be used to detect
that qcow2 is not valid and hence the clone feature is not
available.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-27 08:50:23 +01:00
6c25dbd495 base plugin: get_subdir_files: split stat variables into single lines
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-06 19:27:24 +01:00
c05b1a8cb9 PBS plugin: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-06 19:26:45 +01:00
d65590d1be LVM list_images: return creation time
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-03-06 19:26:45 +01:00
51eee96d31 base plugin: return ctime for vm images
Changed file_size_info() to additionally return ctime to avoid
another stat() call.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-03-06 19:26:45 +01:00
ff9c5451a5 base plugin: add ctime for all files
Creation time makes sense for other file types also.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-03-06 19:26:45 +01:00
545e127e52 PBS Plugin: list_volumes: add ctime
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-03-06 19:26:45 +01:00
9c629b3e76 base plugin: add ctime for backup files
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-03-06 19:26:45 +01:00
553c9b21a7 iscis: add iscsi_session helper
allows to write some code sligthly nicer

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-03 11:33:34 +01:00
c29bad0d90 iscsi: sort and split module usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-03 11:33:34 +01:00
4f430a43ba ISCSI: whitespace cleanup
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-03-03 11:33:34 +01:00
42b988f735 fix #2620: storage API: iSCSI: return active field as integer
If active, the return value was a string: "1" and not an integer.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-03-03 11:33:34 +01:00
93afc379a3 followup: fix VMID regex, use same as JSONSchema does
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-21 16:02:23 +01:00
1b39642528 list_volumes: try to return vmid also for backups
this way the content listing api also returns the vmid on content
listings which, among other things, is useful for the gui for
filtering

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-02-21 16:00:58 +01:00
f33533d4da cifs: followup fix for credential fallback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-20 13:56:18 +01:00