These tests no longer work due to sync only being started after an
account has been added. Functionality of these tests is covered in the
bridge unit tests.
There is concurrency bug due to competing sync calls that can occur when
we clear the sync status in the Vault. Running sync at the end of
addIMAPUser() avoids the problem.
This patch also remove the execution of a sync task for
`user.ClearSyncStatus()`
There was an issue where new attachment download requests would hang
forever due to not checking whether the context was cancelled. At this
point there were no more workers to consume to channel messages.
Update UIDValidity to be timestamp with the number of seconds since
the 1st of February 2023. This avoids the problem where we lose the
last UIDValidity value due to the vault being missing/corrupted/deleted.
Feature was not restored in previous MR. Attachment are now download in
parallel. There is a pool of maxParallelDownloads attachment downloaders
shared with all message downloads.
Updates go-proton-api and Gluon to includes memory reduction changes and
modify the sync process to take into account how much memory is used
during the sync stage.
The sync process now has an extra stage which first download the message
metada to ensure that we only download up to `syncMaxDownloadRequesMem`
messages or 250 messages total. This allows for scaling the download
request automatically to accommodate many small or few very large
messages.
The IDs are then sent to a download go-routine which downloads the
message and its attachments. The result is then forwarded to another
go-routine which builds the actual message. This stage tries to ensure
that we don't use more than `syncMaxMessageBuildingMem` to build these
messages.
Finally the result is sent to a last go-routine which applies the
changes to Gluon and waits for them to be completed.
The new process is currently limited to 2GB. Dynamic scaling will be
implemented in a follow up. For systems with less than 2GB of memory we
limit the values to a set of values that is known to work.
For every update sent to gluon wait and check the error code to see if
an error occurred.
Note: Updates can't be inspect on the call site as it can lead to
deadlocks.
Sub folders with more than 2 levels of depth (e.g.: Folders/first/second)
could not be created since we did not update the known label list we use
to validate the request.
Prevent infinite error loop in event parsing by not returning errors if
we already have or do not have a given address. This occurs since we
sync the latest state at Bridge startup but still receive the events
which contain these changes later.
Do not treat unknown label creation/deletion/update or deletion in Bridge
as an error as the Gluon cache still needs to receive these events to
correct its internal state.
When calculating the hash for the body to match against sent email to
avoid duplicate addition to the sent folder, do not decode the actual
contents of the body.
It is possible that certain attachments are not formed correctly but
can still accepted by the backend. Trimming spaces and \r characters is
enough to hash the message and match it later on.
This also speeds the process up as we no longer have to perform
encoding conversions.
During sync a user may continue to perform operations on the server it
is possible we run into a message which has a labelID we are not aware
of. To counter this we issue `CreateMessage` updates with
`IgnoreUnknownMailboxIDs` set to true. Eventually, after sync the state
will resolve itself with events.
