If for some reason all the message we are trying to sync in a chunk are
deleted from another client it is possible that the input to the build
stage will be empty. This case is now handled correctly.
Improve the handling of sync child jobs to ensure it behaves correctly
in all scenarios.
The sync service now uses a isolated context to avoid all the pipeline
stages shutting down before all the sync tasks have had the opportunity
to run their course.
The job waiter now immediately starts with a counter of 1 and waits
until all the child and the parent job finish before considering the
work to be finished.
Finally, we also handle the case where a sync job can't be queued
because the calling context has been cancelled.
Fix incorrect use of `sync.WaitGroup` use to wait on sync jobs that
fail. After calling `WaitGroup.Wait()` it is advised to call
`WaitGroup.Add` until the existing counter has reached 0.
The code has been updated with a different mechanism that achieves the
same behavior which was previously available.
* Fix wrong context use for message downloads
* Fix delete of sync data failing due ErrNotFound
* Pre-allocate attachment data buffer before download
* Fix calculation of progress if message count is higher than total
Fix tracking of child jobs. The build stage splits the incoming work
even further, but this was not reflected in the wait group counter. This
also fixes an issue where the cache was cleared to late.
Add more debug info for analysis.
Refactor sync state interface in order to have persistent sync rate.
Implementation of the new sync service that interleaves syncing jobs for
all active users.
It also includes improvements to the message downloader. The download
will now auto rate limit the parallel workers based on the server
responses.
Additionally each of the stages is now tested in isolation to ensure the
behavior matches the expectations.
Finally, this patch does not replace the existing IMAP sync. A follow up
patch is necessary to integrate the IMAP bits into the interfaces
required by these changes.
