Make sure that we are at least using 16 workers for sync, otherwise
multiply the current sync worker count by 2.
Finally, this patch also logs the duration of the time it takes to
transfer all the messages from the server.
Revise syncing work distribution. Sync time can be reduced by up to 50%.
Rework the sync so that it pipelines better with bigger batch counts at
each stage. We now use 3 separate stages: Download, Updates and Sync.
The Download stage downloads messages in maxBatchSize intervals using
1.5x syncWorkers. Once the current batch has finished downloading it's
forwarded to the Updates stage and we proceed to download the next
batch.
The Update stage converts everything into gluon updates and prepares a
collection of noops that the sync stage can wait on for termination.
Finally the sync stage waits until the updates have been applied in
Gluon so that the vault information can be updated. We allow up to 4
pending wait operations to be queued currently to not block the
pipeline.
When we send a message, the send recorder records the sent message.
When the client then appends an identical message to the sent folder,
the deduplication works and instead returns the message ID of the
existing proton message, rather than creating a new message. Gluon is
expected to notice that it already has this message ID and perform
some deduplication stuff internally.
However, it can happen that gluon doesn't yet have this message ID,
because we haven't yet received the "Message Created" event from the
API. To prevent this, we poll the events after send and wait for all
new events to be applied.
There's still a chance that the event wasn't generated yet on the API
side. Not sure what we can do about this.
We now set disposition during attachment upload. However, this presents
a problem: some clients use inline disposition but don't provide a
content ID. Our API doesn't support this. To mitigate the issue, we just
fall back to attachment disposition in this case.
Some messages were not being deleted properly because they were also
present in the All-Sent folder.
The code has now been changed to filter out AllMail, AllDraft and
AllSend. If there are no remaining labels, the message will be deleted
permanently.
Add special case handling for draft messages so that if a Draft is
updated via an event it is correctly updated on the IMAP client via a
the new `imap.MessageUpdated event`.
This patch also updates Gluon to the latest version.
After sending, a client might append to the sent folder over IMAP.
In this case, we perform deduplication and return the message ID of the
sent message. However, if we haven't already processed this message in
gluon, it doesn't work as expected.
This change polls the event stream immediately after send. Note that it
doesn't wait for these events to be processed; that should be done in a
follow-up commit.
The newer liteapi contact code saves keys as *crypto.Key, however the
legacy code expects them to be strings. There was a bug in connecting
the new code to the legacy code: it was assumed these strings were meant
to be a base64 encoded string but they were actually just raw string
bytes.
Only delete messages when unlabeled from trash/spam if they only exists
in All Mail and (spam or trash).
This patch also ports delete_from_trash.feature and use status rather
than fetch to count messages in a mailboxes.
