Commit Graph

232 Commits

Author SHA1 Message Date
c43655d2ed vdisk_list: skip scanning storages which cannot have images/rootdisks
Do not try to scan (and thus activate) storages which aren't
configured to support (or cannot support) "vdisks" anyway.

Avoids seemingly strange failures of VM migrations due to a backup storage
not being currently online - even if that storage isn't referenced in
the VM config anywhere..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 16:16:23 +02:00
28be2a431b archive_info: relax custom name handling
we already differentiate between standard and non-standard names anyway
and don't detect and return the VMID in the latter case anyway. drop it
from the RE as well to allow names like 'vzdump-qemu-template.vma.lzo'
without the need for a fake VMID.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-07-08 15:26:19 +02:00
b1ddc54a93 archive_info: use timelocal correctly
Because we always have 4-digit years, we can simply pass
the year itself to timelocal instead of subtracting 1900.
Like this it will also work for years not in the range 2000-2999.

See also:
https://perldoc.perl.org/Time/Local.html#Year-Value-Interpretation

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-08 10:45:49 +02:00
d37729b95b scan_cifs: fix scanning server with no SMB1 fallback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 15:46:30 +02:00
8594155a66 scan cifs: fix passing "no pass" parameter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 12:17:02 +02:00
c42d86559c scan_cifs: do not add NT_STATUS lines to result
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 12:11:59 +02:00
67a2371180 scan_cifs: do not enforce password for users
there can be accounts with explicit null password others than the
mapped guest account.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 12:11:59 +02:00
afaa98f10f scan_cifs: pass user/pass over environment
As command line argument they are readable by ever user in the same
PID namespace.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 12:06:53 +02:00
fd55f51efa scan_cifs: add workgroup domain independent of user param
they are not really bound to each other

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 12:06:53 +02:00
be18d6dac7 scan_cifs: raise smbclient errors
Silencing error messages can extend the search for an issue while
having to advantage.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 12:06:53 +02:00
436773fd78 scan_cifs: fix comment typo and indentation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-01 12:06:53 +02:00
9afd3be18c add trailing newline to error message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-30 14:19:24 +02:00
35a3953213 Add archive_remove
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-30 13:59:23 +02:00
e34afeb117 Extend archive_info to include filename and logfilename
Only expect the logfilename if the archive has a standard name.
This also gives a mechanism to get an untainted filename.

archive_info can take either a volume ID or a path as it's
currently implemented. This is useful for vzdump when there
is no storage (i.e. for 'vzdump --dumpdir'). Add a test case for this.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-30 13:59:23 +02:00
12442ae19c Fix 2763: Revert "storage_migrate: check if target storage supports content type"
This reverts commit 95015dbbf2.

parse_volname always gives 'images' and not 'rootdir'. In most
cases the volume name alone does not contain the needed information,
e.g. vm-123-disk-0 can be both a VM volume or a container volume.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-23 14:43:17 +02:00
fb821c1828 Expand archive_info to include ctime, vmid and is_std_name
where 'is_std_name' shows whether the backup name uses the standard naming
schema and most likely was created by our tools.

Also adds a '^' to the existing filename matching regex, which
should be fine since basename() is used beforehand.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-06 19:38:02 +02:00
5029f978c3 archive info: keep some basic strictness
we want to enforce at least the strictness that our tools can do
something with a backup archive..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:18:24 +02:00
bf5af0fbf5 archive info: include archive name in error message
so that we have some context if users report issues with it..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 19:13:39 +02:00
63e892950b storage: archive/extract code cleanup
* run_command is already imported, use that fact
* avoid useless comments just describing what the code tells one
  anyway
* restructure a few parts to more concise/easier to read
  implementation.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-15 18:46:17 +02:00
4b26f8140d Fix: backup: relax file name matching regex
The rework of the backup file detection logic missed the non-standard
file name case. This patch allows to restore backups with different file
names. Though the config extraction fails, since the type is unknown.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-05-15 18:12:01 +02:00
014d36dbbb Fix: #2124 storage: add zstd support
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
277cafc0ff backup: compact regex for backup file filter
the more compact form of the regex should allow easier addition of new
file extensions.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
73fcb7bfd1 Fix: add missing snippets subdir
since it is a valid content type and adapt the path_to_volume_id_test.
Also adds an extra check if all vtype_subdirs are returned.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
892dc99249 Fix: path_to_volume_id returned wrong content
type for backup files. Patch includes changes of the test as well.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
cd554b79d1 storage: test: split archive format/compressor
detection into separate functions so they are reusable and easier
modifiable. This patch also adds the test for archive_info.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-04-30 18:37:19 +02:00
2668adcec2 Add comment for volume_has_feature
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-09 12:05:11 +02:00
3881e68025 ZFS: use -p flag and remove zfs_parse_size
ZFS supports the -p flag in the list command since a few years now.
Let us use the real byte values and avoid the error prone calculation
from human readable numbers that can lead to incorrect numbers if the
reported human readable value is a rounded number.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-04-09 10:20:06 +02:00
683a3f463f storage_migrate: add volname_for_storage helper
to guess a valid volname for a targetstorage of a different type.
This makes it possible to migrate raw volumes between 'dir' and 'lvm'
storages.

It is only used when the storage type for the source storage X
and target storage Y differ and should work as long as Y uses
the standard naming scheme (VMID/vm-VMID-name.fmt respectively vm-VMID-name).
If it doesn't, we get an invalid name and fail, which is the old
behavior (except if X and Y have different types but the same
non-standard naming-scheme, where the old behavior did work).

The original name is preserved, except for a possible extension
and it's also checked if the format is valid for the target storage.
Example: mylvm:vm-123-disk-4 <-> mydir:123/vm-123-disk-4.raw

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-09 09:41:01 +02:00
a97d3ee49f Introduce allow_rename parameter for pvesm import and storage_migrate
and also return the ID of the allocated volume. This option
allows plugins to choose a new name if there is a collision.

In storage_migrate, the API version of the receiving side is checked.

In Storage.pm's volume_import, when a plugin returns 'undef',
it can be assumed that the import with the requested volid was
successful (it should've died otherwise) and so volid is returned.
This is done for backwards compatibility with foreign plugins.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-09 09:41:01 +02:00
dc3655a1a5 Collect optional parameters for storage_migrate into $opts
Sanitizing $with_snapshots is done on extraction to save a line.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-09 09:41:01 +02:00
95015dbbf2 storage_migrate: check if target storage supports content type
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-27 14:04:09 +01:00
e6f4eed435 Allow passing options to volume_has_feature
With the option valid_target_formats it's possible
to let the caller specify possible formats for the target
of an operation.
[0]: If the option is not set, assume that every format is valid.

In most cases the format of the the target and the format
of the source will agree (and therefore assumption [0] is
not actually assuming very much and ensures backwards
compatability). But when cloning a volume on a storage
using Plugin.pm's implementation (e.g. directory based
storages), the result is always a qcow2 image.

When cloning containers, the new option can be used to detect
that qcow2 is not valid and hence the clone feature is not
available.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-03-27 08:50:23 +01:00
c72155735d volume_resize: align size to 1 KiB
1. Avoids the error
qemu-img: The new size must be a multiple of 512
for qcow2 disks.
2. Because volume_import expects disk sizes to be a multiple of 1 KiB.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-02-19 15:24:09 +01:00
c855ac150c implement extract_vzdump_config for PBSPlugin 2020-02-19 14:00:04 +01:00
271fe39460 PVE/Storage/PBSPlugin.pm: start new proxmox backup server plugin 2020-02-19 14:00:04 +01:00
8e55b4f288 storage_migrate: only set errfunc for send stream
since we redirect the output to our (insecure) socket, logfunc is only
used for STDERR anyway, so we might as well make it explicit on the
caller side.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-02-05 09:37:09 +01:00
aca83310dd storage_migrate: also log with an insecure connection if there is a log function
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-02-05 08:50:14 +01:00
94dd8b0cfa Remove unused string
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-02-05 08:50:14 +01:00
481f6177a5 grammar fix: s/does not exists/does not exist/g
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-13 12:11:26 +01:00
65bb98596c use PVE::SSHInfo
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:03:00 +01:00
2f08fb4bac use PVE::DataCenterConfig
to make sure that the corresponding cfs_read_file works() works.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:03:00 +01:00
6ed43d81dc cleanup: omit module prefix for call to methods in same module
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-31 07:36:27 +01:00
cfdffd8a20 Actually use target_volid
Migration with --targetstorage was broken because of this.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-17 09:28:52 +02:00
b295e05ebe followup: just use case sensitive flag, no manual hack
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-03 09:42:15 +02:00
fd9ca08c70 fixup: use our to make $iso_extension_re value available for other modules
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-09-03 09:41:50 +02:00
4c69349140 fix #2216: Allow .img files in 'iso' type storages
To maintain full (backwards) compatibility, leave the type name as
'iso' - this makes this patch work without changing every consumer of
storage APIs.

Note that currently these files can only be attached as a CDROM/DVD
drive, so USB-only images can be uploaded but might not work in VMs.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-09-03 07:09:28 +02:00
be78543909 fix 2317: list images on 'rootdir' only storages
plugins can still override list_volumes if they want separate methods to
list rootdir and images content.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-08-06 13:39:43 +02:00
c2fc9fbe1d storage plugin: new list_volumes plugin method
This cleanup improve code reuse, and allows plugins to override list_volumes.
2019-07-19 13:49:29 +02:00
228e5be920 pvesm import: allow to pass a tcp://<IP> as file
To allow getting closer to finally drop "pvecm mtunnel".
Code parts taken from pipe_socket_to_command

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
[regex fixup]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-11 10:13:37 +02:00
5f73686fd3 fix #2226: ignore broken symlinks in *_list
Broken symlinks (and other files without a size) will now show up as 0
byte instead of causing a format validation error in the API.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-06-17 11:49:33 +02:00