Commit Graph

160 Commits

Author SHA1 Message Date
cd461a5012 storage: rename REs for iso and vztmpl extensions
these changes make it more clear, how many capture groups each
RE inclues.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2022-03-16 17:13:59 +01:00
7f30857519 fix #3894: cast 'size' and 'used' to integer
Perl's automatic conversion can lead to integers being converted to
strings, for example by matching it in a regex.

To make sure we always return an integer in the API call, add an
explicit cast to integer.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Reviewed-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-21 16:07:27 +01:00
cde43c4880 api: disks: delete: add flag for cleaning up storage config
Update node restrictions to reflect that the storage is not available
anymore on the particular node. If the storage was only configured for
that node, remove it altogether.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>

slight style fixup

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-10 12:35:25 +01:00
ecfe25058b prune: mark renamed and protected backups differently
While it makes no difference for pruning itself, protected backups are
additionally protected against removal. Avoid the potential to confuse
the two. Also update the description for the API return value and add
an enum constraint.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-08 14:56:15 +01:00
56897a9203 fix #3307: make it possible to set protection for backups
A protected backup is not removed by free_image and ignored when
pruning.

The protection_file_path function is introduced in Storage.pm, so that
it can also be used by vzdump itself and in archive_remove.

For pruning, renamed backups already behaved similiar to how protected
backups will, but there are a few reasons to not just use that for
implementing the new feature:
1. It wouldn't protect against removal.
2. It would make it necessary to rename notes and log files too.
3. It wouldn't naturally extend to other volumes if that's needed.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-08 14:56:15 +01:00
f1de828166 add generalized functions to manage volume attributes
replacing the ones for handling notes. To ensure backwards
compatibility with external plugins, all plugins that do not just call
another implementation need to call $class->{get, update}_volume_notes
when the attribute is 'notes' to catch any derived implementations.

This is mainly done to avoid the need to add new methods every time a
new attribute is added.

Not adding a timeout parameter like the notes functions have, because
it was not used and can still be added if it ever is needed in the
future.

For get_volume_attribute, undef will indicate that the attribute is
not supported. This makes it possible to distinguish "not supported"
from "error getting the attribute", which is useful when the attribute
is important for an operation. For example, free_image checking for
protection (introduced in a later patch) can abort if getting the
'protected' attribute fails.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-11-08 14:56:15 +01:00
855915b60d status: fix tmpfile cleanup
$tmpfilename already gets unlinked after executing the cmd.

furthermore, because this is a local file, it is wrong to delete
it via the ssh command on a remote node.

small change: added \n to the error message.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-10-04 10:22:06 +02:00
0246225ca7 fix #3505: status: add checksum and algorithm to file upload
Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-10-04 10:22:06 +02:00
e4d56f096e status: remove sleep(1) in file upload
this racey sleep(1) is only there for legacy reasons: because
we don't use apache anymore and only emulate its behabiour
regarding removing temp files, this is under our own control
now and so we can improve this whole situation.

this change requires a pve-http-server version, in which the
tmpfile gets not automatically removed anymore.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-10-04 10:22:06 +02:00
3de1c50a89 api: followup style/comment improvements
try to comment why not what, what is already described good enough by
the code here.

Also, we want to go up to 100cc text-width if it improves
readability, which for post-if's it most often does.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-06 08:32:20 +02:00
f6aeefff54 status: move unlink from http-server to enpoint
this is the first step in which not the http server removes the
temporary file, but the worker itself.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-09-06 08:28:26 +02:00
22b68016f7 Ceph: add keyring parameter for external clusters
By adding the keyring for RBD storage or the secret for CephFS ones, it
is possible to add an external Ceph cluster with only one API call.

Previously the keyring / secret file needed to be placed in
/etc/pve/priv/ceph/$storeID.{keyring,secret} manually.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-08-26 18:15:30 +02:00
b11d054be0 api: status: rework err cleanup
avoid open3 on local node.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-29 17:12:40 +02:00
a84804c432 api: status: import run_command
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-29 17:12:40 +02:00
31bd43648d api: status: fix unlink on file upload
after an error while copying the file to its destination the local
path of the destination was unlinked in every case, even when on the
destination was copied to via scp.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-07-29 17:12:40 +02:00
0a65c237ea api: status: sort/cleanup module use-statements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-29 17:12:40 +02:00
ca8c865871 status: add max length and disclaimer to file upload's filename
Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-07-21 16:47:01 +02:00
1e96ffc954 status: add enum for file upload content type
the addition of this enum does not change API behaviour, because
it is checked for 'iso' or 'vztmpl' aftwerwards anyway.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-07-21 16:47:01 +02:00
fc015f3d9c api: file download: limit filename length and encode it for the worker id
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 22:40:40 +02:00
d0a3db1bf5 api: file download: code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 22:40:05 +02:00
837b194224 status: add download_url method
uses common function PVE::Tools::download_file_from_url to download
iso files.

Only users with permissions `Sys.Audit` and `Sys.Modify` on `/` are
permitted to perform this action. This restriction is due to the
fact, that the download function is able to download files from
internal networks (which are not visible/accessible from outside).
Users with these permissions anyway have the means to alter node
(network) config, so this does not create any further security risk.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-06-23 22:28:44 +02:00
edda43ed4f status: factoring out normalize_content_filename
Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-06-23 22:28:44 +02:00
0153334270 api: content: correctly handle warnings status for delayed task
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 22:21:55 +02:00
bba10cf4af factoring out regex for vztmpl
stores the regex definition in PVE::Storage.

One test had to be adapted because it tested obsolete code. Namely:
it expects vztmpl to only end with .tar.gz, but the new regex also
includes .tar.xz, there is nothing against allowing .tar.xz files as
vztmpl files.

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
2021-06-23 20:19:09 +02:00
ffc31266da tree-wide: fix typos with codespell
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 08:28:48 +02:00
ead6be934d api: status: sort index and add missing "file-restore"
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-21 09:32:55 +02:00
f985f33afd api: content/delete: die with newline to avoid addign file-context
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-16 19:24:38 +02:00
0e30b3121d api: get rid of moved 'usb' call
pve-manger commit bd328734deb1dcea296858bb38d085e392adb99e changed the frontend
to use the new call.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-08 15:19:36 +02:00
c1ec1acbde file-restore: pass in volume ID or name
instead of just the snapshot for consistency with other API endpoints,
and possible future extension to VMA backups (where 'snapshot' would be
a rather strange terminology).

add some additional checks (pbs storage type, backup volume type),
completion and magic (allow passing in either a full volume ID with
correct storage, or just the volume name, or just the snapshot for
easier API/CLI usage/convenience).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-23 14:09:54 +02:00
82f764e119 file-restore: return perl-y booleans
like we do in most of our API.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-23 14:09:54 +02:00
f1a3ce3b17 add FileRestore API for PBS
Includes list and restore calls.

Requires VM.Backup and Datastore.Audit permissions, for the accessed
VM/CT and containing datastore respectively.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-23 14:09:48 +02:00
6035a5dfb1 api: fix typo in error message
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-23 12:29:36 +02:00
c56f7a71af pbs: allow setting up a master key
similar to the existing encryption key handling, but without
auto-generation since we only have the public part here.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-22 21:56:31 +02:00
acc9c3acbc api: storage/config: use extract_sensitive_params from tools
we have a more general version there

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 17:31:19 +01:00
f244e2aa7f api: content/backup: handle deletion of notes
Previous to this we did not called the plugins update_volume_notes at
all in the case where a user delted the textarea, which results to
passing a falsy value ('').

Also adapt the currently sole implementation to delete the notes field
in the undef or '' value case. This can be done safely, as we default
to returning an empty string if no notes file exists.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-07 16:10:09 +01:00
405b247168 api: scan: note that USB is depreacated
It now got moved in /nodes/<node>/hardware/usb as envisioned[0], this
allows to sunset the usb scan API endpoint here and drop it with 7.0

[0]: https://lists.proxmox.com/pipermail/pve-devel/2018-November/034694.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-02 11:31:06 +01:00
39fc0c77e2 api/cli: add pbs scan endpoint and command
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
b1ccf4f357 api: scan: move over index and usb scan from manager
Add the missing pieces allowing pve-manager to just point the
/nodes/<node>/scan api directory at this module, dropping it's
duplicated copy.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
e65abdb458 api: scan cifs: port over NT_STATUS filter from pve-manager
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
2892b6739d factor out scan CLI definition to real API module
we have a 1:1 copy of that code in pve-manager's PVE::API2::Scan,
which we can avoid by using a common module form pvesm CLI and the
API.

This is the first basic step of dropping the code duplication in
pve-manager.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
878fe0177e api: content: pass encrypted status for PBS backups
Prefer the fingerprint, fallback to checking the files crypt-mode.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 23:18:05 +01:00
e9991d2694 Storage/Plugin: add get/update_volume_comment and implement for dir
and add the appropriate api call to set and get the comment
we need to bump APIVER for this and can bump APIAGE, since
we only use it at this new call that can work with the default
implementation

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
3369651885 api2/storage/content: change to volume_size_info and add return properties
'file_size_info' only works for directory based storages, while
'volume_size_info' should work for all

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
6fef456c8d rename comment to notes
so that we are more consistent with pbs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
cd69cedf3e api: storage create/update: return parts of the configuration
First, doing such things can make client work slightly easier, as the
submitted values do not need to be made available in any callback
handling the response.

But the actual reason for doing this now is, that this is a
preparatory step for allowing the user to download/print/.. an
autogenerated PBS client encryption key.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 18:01:43 +01:00
9778e5c216 api: content listing: add comment and verification fields
for now only for PBS, since we do not have such info elsewhere

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-12 17:16:55 +01:00
8ff8e27713 api/config: fix indentation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 09:35:53 +01:00
7b73d327b5 prune_backups: fix message
For prune selections, it doesn't matter what the current time is,
only the timestamps of the backups matter.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-08-20 17:28:07 +02:00
25a9583634 Add API and pvesm call for prune_backups
For the pvesm call use a wrapper and a --dry-run option to redirect
to the correct API call.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
b3b63fc27d fix another encryption_key vs encryption-key instance
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 13:51:26 +02:00