Commit Graph

166 Commits

Author SHA1 Message Date
77445e9b80 bwlimit: apply limits when there's no override
Before, 'undef' was equivalent to unlimited, but '0' is the
"explicitly unlimited" value, so if the user doesn't request
an override, apply limits as if the user was unprivileged
(otherwise there's no way for privileged users to explicitly
ask to not override the configured limits).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-03-07 14:19:07 +01:00
9edb99a5a7 add Storage::get_bandwidth_limit helper
Takes an operation, an optional requested bandwidth
limit override, and a list of storages involved in the
operation and lowers the requested bandwidth against global
and storage-specific limits unless the user has permissions
to change those.
This means:
 * Global limits apply to all users without Sys.Modify on /
   (as they can change datacenter.cfg options via the API).
 * Storage specific limits apply to users without
   Datastore.Allocate access on /storage/X for any involved
   storage X.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-01-31 12:25:32 +01:00
856c54bd75 add 'format' parameter to storage list call
we will use this for the gui to figure out if we have to show
a size selector, a file selector, which formats are there, etc.

we have to include this data even for not active storages, else
we cannot show the correct fields

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-11-14 13:23:52 +01:00
8fe00d9944 Add logfung in storage_migrate
The log function is used by storage replication to redirect the output in the correct log.
2017-10-16 14:59:57 +02:00
6ce4f72487 cleanup: call defined() early
No need to carry around a hash we don't need if we get one.
2017-08-10 13:24:41 +02:00
b43b073b51 Fix #1474: Fixes input and output of print_storage
Accommodates changes in 44ae567 and d40e27d by
reordering checks to allow for proper filtering
of disabled storages. Also reorders two checks to
prevent autovivification resulting in disabled
storages always showing in output.
2017-08-10 13:20:52 +02:00
44ae567c38 storage_info: return also disabled storages
in the Storage/Status API call we have a 'enabled' param which had no
effect because storage_info only returned enabled one way or the
other.

This affected also `pvesm status` which uses the Storage/Status API
call.

So push also disabled storages to the info array but only activate
and get their status when thei are enabled.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-07-27 13:09:28 +02:00
7118dd916b PVE::Storage::storage_can_replicate - hew helper 2017-06-27 06:17:58 +02:00
e8a7e764af storage_migrate: with_snapshots parameter
Since the migration code already has to check snapshots and
we still want to be able to migrate raw disks, we cannot
always pass -with-snapshots 1.
2017-06-22 12:40:23 +02:00
da72898cc6 migrate: only use import/export
This replaces the path-based and lvm/thin special cases in
storage_migrate with the already generic-enough zfspool
case which is already using import/export and does not
directly depend on zfs anymore.
2017-06-22 07:03:39 +02:00
ae36189d26 export: add missing format query call parameter 2017-06-22 05:51:32 +02:00
488cf14d4f migrate: insecure: untaint the received ip/port 2017-06-09 12:30:59 +02:00
47cea19432 migrate: don't include the hostname in rsync's rsh command 2017-06-09 12:30:17 +02:00
4b4c580d02 migrate: use the correct IP with rsync 2017-06-09 12:30:02 +02:00
636ac5b82f PVE::Storage::volume_snapshot_list - remove comment about ordering
Some storage types supports arbitrary snapshot trees, so there is
no strict ordering relation.
2017-06-07 06:36:55 +02:00
8b622c2dff PVE::Storage::volume_snapshot_list - remove $prefix parameter
Always return the full list of snapshots. Users of this library can easily
filter with a simply 'grep' instead.
2017-06-07 06:20:07 +02:00
7ba34faa6e migrate: add insecure parameter
This uses mtunnel's new -run-command parameter in
combination with the ssh-info's network cidr option to use
an unprotected TCP port.
2017-06-01 10:43:32 +02:00
01f7e902e9 migrate: implement rate limiting
Note that it does not apply to the local 'mkdir && cp' case.
2017-06-01 10:42:32 +02:00
b57a863a0f migrate: cleanup & fixup
The volume_snapshot call was missing the condition when to
create a snapshot. Make the whole logic easier to follow
with a $migration_snapshot boolean.
Also get rid of the remote `pvesm free -snapshot` call by
using import's new -delete-snapshot parameter.
2017-05-23 13:30:51 +02:00
acd27197ce migrate: replace target ip parameter with sshinfo parameter 2017-05-23 09:31:19 +02:00
dc7695377c migrate: snapshot parameter for use with replication 2017-05-23 09:27:26 +02:00
55f908a1cd migrate: cleanup: $free is always defined 2017-05-12 15:01:08 +02:00
7459cb3d91 migrate: query transfer stream formats 2017-05-12 14:42:17 +02:00
d390328bfd api: add import/export format querying 2017-05-12 14:42:17 +02:00
ac191ec740 migrate: snapshot parameter for incremental transfer 2017-05-12 14:42:17 +02:00
865710f47a migrate: use pvesm import/export for zfs
The only leftover zfs specific part here is the zfs format
type parameter.
2017-05-12 14:42:17 +02:00
47f37b5362 pvesm: import/export commands 2017-05-12 14:42:16 +02:00
0d5a5fc9cb Revert "Include incremental zfs send in storage_migrate."
This reverts commit 4537523b5a.

We want to use send/receive for this - patch will follow.
2017-05-10 07:20:00 +02:00
17be2e9a0c volume_snapshot_list: remove $ip parameter
We want to handle ssh connections somewhere else (not inside the
storage plugins).
2017-05-10 07:02:42 +02:00
3d4949692a Revert "Include new storage function volume_send."
This reverts commit b76774e57f.
2017-05-10 06:58:44 +02:00
889d7485cb Revert "Add function volume_snapshot_delete_remote."
This reverts commit 4bd0b38f53.
2017-05-10 06:55:00 +02:00
4537523b5a Include incremental zfs send in storage_migrate.
When replica is enabled and the target host is the reptarget,
the most VM data are on the new target.
2017-04-28 10:05:27 +02:00
4bd0b38f53 Add function volume_snapshot_delete_remote.
This function we need for replica to handle snapshots on remote nodes.
2017-04-28 10:05:27 +02:00
aefe82ea03 Include new storage function volume_snapshot_list.
Returns a list of snapshots (youngest snap first) form a given volid.
It is possible to use a prefix to filter the list.
2017-04-28 10:05:27 +02:00
b76774e57f Include new storage function volume_send.
If the storage backend support import and export
we can send the contend to a remote host.
2017-04-28 10:05:27 +02:00
fc1089fc3a fix run_command return code handling 2017-04-10 15:29:29 +02:00
086c4bf12b extract_vzdump_config: handle RE consistently
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-01-20 11:14:01 +01:00
8944339466 weaken matching for vzdump config extract filename
to allow "pvesm extractconfig filename" when in the dump
directory.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-01-20 11:13:58 +01:00
04a13668b9 PVE::Storage: new helper check_volume_access()
Copied from PVE::RPCEnvironment to avoid cyclic dependency
(pve-storgae => pve-access-control => pve-storage).
2017-01-19 09:14:41 +01:00
afd0afe4f5 fix #1135: always clean up ZFS snapshots on migration
otherwise there are situations where snapshots are left
behind for already sent volumes. also include more warnings.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-01-03 11:45:16 +01:00
46f114a53f storage_migrate: use bs=4k to copy images with dd
The default block size is 512 bytes, and is quite slow. We do not
want to use blocks bigger than 4k, because we want to detect empty
blocks (sparse).
2016-12-24 10:55:31 +01:00
ff3badd83f white space cleanups 2016-09-26 13:40:43 +02:00
a9ef8ffb16 Avoid JavaScript gets a string 0.
If the JavaScript gets a "0" it convents it to a boolean false.
So to ensure the GUI always get valid int we cast the values.
2016-09-26 13:38:56 +02:00
787624dfc0 add comments about LVM thin clones 2016-09-15 14:01:07 +02:00
17fb7e4215 move check for existing clones into own method
and change its return type to boolean
2016-09-15 13:52:57 +02:00
9924228be1 remove unused method
only used by test case, which should use what the rest of
the codebase uses as well
2016-09-15 13:42:55 +02:00
3718e83ab5 fix error message 2016-09-14 11:24:06 +02:00
4dee23d305 Add support for custom storage plugins
PVE team cannot support specialized vendor-specific storage
plugins because of lack of hardware. But we can allow users to
add own plugins for their storages without need to rewrite any
PVE code and thus ease PVE updates to them.

Idea of this patch is to add folder /usr/share/perl5/PVE/Storage/Custom
where user can place his plugins and PVE will automatically load
them on start or warn if it could not and continue. Maybe we could
even load all plugins (except PVE::Storage::Plugin itself) this way,
because current storage plugins are not really plugins, if they
need to be explicitly loaded in PVE code :-).

Custom plugins MUST have api() method returning version for which
it was designed. If API changes from PVE side, module is just not
being registered and warnig message is printed do log, so user have
to update module. Until module update, corresponding storage will
just disappear from PVE, so it shall not impose any data damage
because of API change.

This approach works (with some limitations) if plugin works in
generic PVE way: full control of volumes lifecycle. And will not
currently work for custom plugins like iSCSI, which needs to select
pre-existing volumes. Maybe someone will add more flexible way to
pve-manager to select input elements for storage plugins to target
this.

Currently tested with my NetApp plugin.

Signed-off-by: Dmitry Petuhov <mityapetuhov@gmail.com>
2016-08-26 15:44:49 +02:00
f3b3b2a3b7 remove compression option from lvm migration
ssh(1) mentions that compression is only disirable on slow
connections.

since migration from cluster node to cluster node needs a
fast network anyway, we can drop the compression for
a speed improvement

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2016-08-05 08:36:13 +02:00
82fc923fd4 fix spelling / grammar 2016-07-13 13:59:26 +02:00