Commit Graph

125 Commits

Author SHA1 Message Date
1e29a5210f GODT-1954: Draft message support
Add special case handling for draft messages so that if a Draft is
updated via an event it is correctly updated on the IMAP client via a
the new `imap.MessageUpdated event`.

This patch also updates Gluon to the latest version.
2022-11-16 13:48:30 +01:00
c548ba85fe Other: Add more extensive logging 2022-11-16 13:48:30 +01:00
bc7912e8fb GODT-2022: Fix change between address modes 2022-11-16 13:48:30 +01:00
af542a2fc1 Other: Ensure logout works when offline 2022-11-16 13:48:30 +01:00
039d1b7f99 GODT-2023: Revert to v2 bridge password encoming format
v2 used base64.RawURLEncoding rather than hex to encode the bridge
password. We should use that in v3 as well.
2022-11-16 13:48:30 +01:00
4ded8784fc Other: Allow non-received messages to be imported to INBOX 2022-11-16 13:48:30 +01:00
75b788b793 GODT-1993: Use more efficient filtering for message deletion 2022-11-16 13:48:30 +01:00
62d62474fb Other: Poll events after each IMAP operation
This matches bridge's behaviour (except for it being a non-blocking
wait).

It also cleans up some unused connector methods.
2022-11-16 13:48:30 +01:00
99745ac067 Other: Switch to faster message IDs route
The new API route lets us query exactly which message IDs a user has,
allowing us to begin syncing much faster than before.
2022-11-16 13:48:30 +01:00
dbfb7572a8 GODT-2008: Ensure user's addresses are returned in sorted order 2022-11-16 13:48:30 +01:00
a213b48f93 GODT-2002: Poll after SMTP send
After sending, a client might append to the sent folder over IMAP.
In this case, we perform deduplication and return the message ID of the
sent message. However, if we haven't already processed this message in
gluon, it doesn't work as expected.

This change polls the event stream immediately after send. Note that it
doesn't wait for these events to be processed; that should be done in a
follow-up commit.
2022-11-16 13:48:30 +01:00
075e1ef236 Other: Fix wrong encoding used for public key during sending
The newer liteapi contact code saves keys as *crypto.Key, however the
legacy code expects them to be strings. There was a bug in connecting
the new code to the legacy code: it was assumed these strings were meant
to be a base64 encoded string but they were actually just raw string
bytes.
2022-11-16 13:48:30 +01:00
a797c01943 Other: Fix create draft action
If an InReplyTo entry in the header is present, the create draft action
should be reply, otherwise we use forward.
2022-11-16 13:48:30 +01:00
b72de5e3a4 Other: Fix goroutine leak in OnStatusDown
We should spawn the goroutine as a bridge async task rather than as a
normal goroutine
2022-11-16 13:48:30 +01:00
8f420d728c GODT-1984: Handle permanent message deletion
Only delete messages when unlabeled from trash/spam if they only exists
in All Mail and (spam or trash).

This patch also ports delete_from_trash.feature and use status rather
than fetch to count messages in a mailboxes.
2022-11-16 13:48:30 +01:00
df818bc2b8 Other: Update UID validity in vault when necessary 2022-11-16 13:48:30 +01:00
3c4e8730ac Other: Read sync workers setting from vault 2022-11-16 13:48:30 +01:00
d066e32719 GODT-1986: Handle case where an address has no decryption entities
It's possible (but very rare, I don't think proton still allows it)
for an address to have no keys. If we try to load the address keyring
for such an address, this change logs a warning that no decryption
entities were found in the unlocked keyring.

It bumps liteapi to a version that does not return an error when no
keys could be unlocked.
2022-11-16 13:48:30 +01:00
e4c08be28e Other: Don't remove/add gluon user when in combined mode
This used to be necessary because the connectors didn't have access
to the user's emails, but now they do, so we no longer need to do this.
2022-11-16 13:48:30 +01:00
7e03de0a21 Other: Lint fixes 2022-11-16 13:48:30 +01:00
d4da325e57 Other(refactor): Sort safe.Mutex types before locking to prevent deadlocks
This change implements safe.Mutex and safe.RWMutex, which wrap the
sync.Mutex and sync.RWMutex types and are assigned a globally unique
integer ID. The safe.Lock and safe.RLock methods sort the mutexes
by this integer ID before locking to ensure that locks for a given
set of mutexes are always performed in the same order, avoiding
deadlocks.
2022-11-16 13:48:30 +01:00
fd80848fcd Other(refactor): Use normal value + mutex for user.updateCh 2022-11-16 13:48:30 +01:00
cab5ee6752 Other(refactor): Use normal value + mutex for user.apiLabels 2022-11-16 13:48:30 +01:00
0bc99dbd4f Other(refactor): Use normal value + mutex for user.apiAddrs 2022-11-16 13:48:30 +01:00
83339da26c Other(refactor): Use normal value + mutex for user.apiUser 2022-11-16 13:48:30 +01:00
2bda47fcad Other(refactor): Less unwieldy user type in Bridge
Instead of the annoying safe.Map type, we just use a normal go map
and mutex pair, and use safe.Lock/safe.RLock as a helper function
2022-11-16 13:48:30 +01:00
85c0d6f837 Other: Hold user labels in memory
Labels can be held locally and updated in memory. This greatly improves
the responsiveness of IMAP mailbox operations as we don't need to fetch
all a user's labels to find the parent whenever a mailbox is moved.
2022-11-16 13:48:30 +01:00
c7a5b8559c Other: Updating address flags should be handled 2022-11-16 13:48:30 +01:00
fd0c262645 Other: Implement subfolder support 2022-11-16 13:48:30 +01:00
14fbdb5e04 Other: Fix race condition when changing address mode
When changing address mode, we would close all a user's update channels
and create them from scratch. This involved setting user.updateCh to a
new value. However, it was possible for other goroutines to read from
user.updateCh during this time. I replaced it with a call to
user.updateCh.Clear(), which is threadsafe.
2022-11-16 13:48:30 +01:00
35fa43f47c Other: Properly handle SMTP to list in send recorder
Checking the BCC header is unreliable; it is usually omitted from messages.
Instead, we can use the SMTP "to" list for deduplication.
2022-11-16 13:48:30 +01:00
4df8ce1b58 Other: Linter to ignore test parameters 2022-11-16 13:48:30 +01:00
83c7396f2d Other: Separate getMessageHash from sendRecorder 2022-11-16 13:48:30 +01:00
709922c383 Other(lint): gofmt 2022-11-16 13:48:30 +01:00
16978b8949 Other(test): Add send recorder test case 2022-11-16 13:48:30 +01:00
7745b68228 GODT-1777: Also include Reply-To and In-Reply-To in message hash 2022-11-16 13:48:30 +01:00
036a416a25 GODT-1777: Message de-duplication in IMAP (+ cleanup) 2022-11-16 13:48:30 +01:00
c9808d07df GODT-1777: Message de-duplication in SMTP 2022-11-16 13:48:30 +01:00
828385b049 Other: Fix user sync leaks/race conditions
This fixes various race conditions and leaks related to the user's sync
and API event stream. It was possible for a sync/stream to begin after a
user was already closed; this change prevents that by managing the
goroutines related to sync/stream within cancellable groups.
2022-11-16 13:48:30 +01:00
6bbaf03f1f Other: Fix goroutine leaks in sync tests
Add missing Close calls.

Properly handle nil channel for `user.startSync`.

This patch also updated liteapi and Gluon to latest master and dev
version respectively.
2022-11-16 13:48:30 +01:00
0f125196a6 Other: Bump go-smtp version to fix race condition
There was a race condition internal to the go-smtp library.
In order to fix it, a version bump was necessary.
However, this significantly changed the library interface.
2022-11-16 13:48:30 +01:00
974735d415 Other: Bump liteapi version to fix goroutine leaks 2022-11-16 13:48:30 +01:00
81f4ef609b Other: Mitigate double-unlock of user keyring
We need to unlock the user keyring anyway to unlock the address keyring,
so we should just return it instead of re-unlocking the user keyring
when sending a message.
2022-11-16 13:48:30 +01:00
80d3f7d179 Other: Set \Draft flag on messages in drafts mailbox 2022-11-16 13:48:30 +01:00
c4343e0124 Other: Bump liteapi and clean up tests a bit 2022-11-16 13:48:30 +01:00
04b6571cb8 Other: Handle Seen/Flagged IMAP flags when APPENDing a message
When an IMAP client appends a message to a mailbox, it can specify
which flags it wants the appended message to have. We need to handle
these in a proton-specific way; not-seen messages need to be imported
with the Unread bool set to true, and flagged messages need to
additionally be imported with the Starred label.
2022-11-16 13:48:30 +01:00
a7a7d9a3d4 GODT-1742: Implement hide All Mail 2022-11-16 12:26:09 +01:00
1729c085c7 Other: Fix user logout hangs due to sync 2022-11-16 12:26:09 +01:00
bf29090ffa Other: Fix log message, rename test fixture 2022-11-16 12:26:09 +01:00
ca132881f9 Other: Update Gluon to v0.13.0 2022-11-16 12:26:09 +01:00