Improve the handling of sync child jobs to ensure it behaves correctly
in all scenarios.
The sync service now uses a isolated context to avoid all the pipeline
stages shutting down before all the sync tasks have had the opportunity
to run their course.
The job waiter now immediately starts with a counter of 1 and waits
until all the child and the parent job finish before considering the
work to be finished.
Finally, we also handle the case where a sync job can't be queued
because the calling context has been cancelled.
Ensure that the heartbeat background task is stopped before we close
the users as it accesses data within these instances.
Additionally, we also make sure that when telemetry is disabled, we stop
the background task.
Finally, `HeartbeatManager` now specifies what the desired interval is
so we can better configure the test cases.
* Ensure IMAP service sync cancel request waits until the sync has
completely cancelled rather than just signaling. It's possible that
due the context reset on `group.Cancel` that something may have not
have been bookmarked correctly in subsequent sync restarts.
* Handle connection lost/restored events in the services. Removes the
need to lock bridge users. Which could conflict with other ongoing
lock operations. Additionally, it ensure that if one service is
blocked it doesn't block the entire bridge.
* Revise access to bridge user locks.
It should not be possible to reach this state on purpose. But due to
scheduling and synchronization variances, it is possible in theory that
a UserDeathEvent can occur at the same time as the bridge is closing,
causing a call to `User.Close()` to be executed 2 times.
To avoid this in the future we just clear the map after all the channels
have been closed.
Add a dedicated go-routine whose sole responsibility is to manage the
life time of the IMAP and SMTP servers and their listeners.
The current implementation behaves the same way as the previous state.
The new behavior will be implemented in a follow MR.
We don't want to start processing events until those events have
somewhere to be sent to.
Also, to be safe, ensure remove and re-add the gluon user while
clearing its sync status. This shouldn't be necessary.
fix(GODT-2327): Only start processing events once sync is finished
fix(GODT-2327): avoid windows delete all deadlock
fix(GODT-2327): Clear update channels whenever clearing sync status
fix(GODT-2327): Properly cancel event stream when handling refresh
fix(GODT-2327): Remove unnecessary sync abort call
fix(GODT-2327): Fix lint issue
fix(GODT-2327): Don't retry with abortable context because it's canceled
fix(GODT-2327): Loop to retry until sync has complete
fix(GODT-2327): Better sleep (with context)
Update UIDValidity to be timestamp with the number of seconds since
the 1st of February 2023. This avoids the problem where we lose the
last UIDValidity value due to the vault being missing/corrupted/deleted.
Updates go-proton-api and Gluon to includes memory reduction changes and
modify the sync process to take into account how much memory is used
during the sync stage.
The sync process now has an extra stage which first download the message
metada to ensure that we only download up to `syncMaxDownloadRequesMem`
messages or 250 messages total. This allows for scaling the download
request automatically to accommodate many small or few very large
messages.
The IDs are then sent to a download go-routine which downloads the
message and its attachments. The result is then forwarded to another
go-routine which builds the actual message. This stage tries to ensure
that we don't use more than `syncMaxMessageBuildingMem` to build these
messages.
Finally the result is sent to a last go-routine which applies the
changes to Gluon and waits for them to be completed.
The new process is currently limited to 2GB. Dynamic scaling will be
implemented in a follow up. For systems with less than 2GB of memory we
limit the values to a set of values that is known to work.
