Commit Graph

753 Commits

Author SHA1 Message Date
b1ce2dd73f Other: QA host URL when built with build_qa tag 2022-11-16 13:48:30 +01:00
d066e32719 GODT-1986: Handle case where an address has no decryption entities
It's possible (but very rare, I don't think proton still allows it)
for an address to have no keys. If we try to load the address keyring
for such an address, this change logs a warning that no decryption
entities were found in the unlocked keyring.

It bumps liteapi to a version that does not return an error when no
keys could be unlocked.
2022-11-16 13:48:30 +01:00
7f6094750e Other: Use logrus for liteapi logs so we can properly capture them 2022-11-16 13:48:30 +01:00
e4c08be28e Other: Don't remove/add gluon user when in combined mode
This used to be necessary because the connectors didn't have access
to the user's emails, but now they do, so we no longer need to do this.
2022-11-16 13:48:30 +01:00
7e03de0a21 Other: Lint fixes 2022-11-16 13:48:30 +01:00
d4da325e57 Other(refactor): Sort safe.Mutex types before locking to prevent deadlocks
This change implements safe.Mutex and safe.RWMutex, which wrap the
sync.Mutex and sync.RWMutex types and are assigned a globally unique
integer ID. The safe.Lock and safe.RLock methods sort the mutexes
by this integer ID before locking to ensure that locks for a given
set of mutexes are always performed in the same order, avoiding
deadlocks.
2022-11-16 13:48:30 +01:00
5a4f733518 Other(refactor): Remove unused safe types 2022-11-16 13:48:30 +01:00
fd80848fcd Other(refactor): Use normal value + mutex for user.updateCh 2022-11-16 13:48:30 +01:00
cab5ee6752 Other(refactor): Use normal value + mutex for user.apiLabels 2022-11-16 13:48:30 +01:00
0bc99dbd4f Other(refactor): Use normal value + mutex for user.apiAddrs 2022-11-16 13:48:30 +01:00
83339da26c Other(refactor): Use normal value + mutex for user.apiUser 2022-11-16 13:48:30 +01:00
8749d5dc7d Other(refactor): Remove always-nil return value of (*Bridge).Close 2022-11-16 13:48:30 +01:00
2bda47fcad Other(refactor): Less unwieldy user type in Bridge
Instead of the annoying safe.Map type, we just use a normal go map
and mutex pair, and use safe.Lock/safe.RLock as a helper function
2022-11-16 13:48:30 +01:00
85c0d6f837 Other: Hold user labels in memory
Labels can be held locally and updated in memory. This greatly improves
the responsiveness of IMAP mailbox operations as we don't need to fetch
all a user's labels to find the parent whenever a mailbox is moved.
2022-11-16 13:48:30 +01:00
04d9fa8f9e Other: Fix apple mail config 2022-11-16 13:48:30 +01:00
7b7a2068ea Other: Clear keychain entries on factory reset 2022-11-16 13:48:30 +01:00
784896434d Other: Add vault editor CI tool 2022-11-16 13:48:30 +01:00
d376b88cf0 Other: Use msgpack instead of json in vault
msgpack is much faster at serializing and deserializing than json. Using
it in the vault gives a performance boost.
2022-11-16 13:48:30 +01:00
c7a5b8559c Other: Updating address flags should be handled 2022-11-16 13:48:30 +01:00
fd0c262645 Other: Implement subfolder support 2022-11-16 13:48:30 +01:00
14fbdb5e04 Other: Fix race condition when changing address mode
When changing address mode, we would close all a user's update channels
and create them from scratch. This involved setting user.updateCh to a
new value. However, it was possible for other goroutines to read from
user.updateCh during this time. I replaced it with a call to
user.updateCh.Clear(), which is threadsafe.
2022-11-16 13:48:30 +01:00
5adbf74cbe Other(test): Ensure calls are protected by mutex 2022-11-16 13:48:30 +01:00
7df2d70dbf Other: Fix race condition in bridge mock updater
We need to protect the latest version info by a lock.
2022-11-16 13:48:30 +01:00
350544e801 Other: Fix race conditions in internal/dialer
Some race conditions came from the tests themselves. But we had a race
condition reading the proxyAddress; this change protects it with a
mutex.
2022-11-16 13:48:30 +01:00
d0fb3509cc Other: Make user-agent implementation threadsafe 2022-11-16 13:48:30 +01:00
798cd5caf3 Other: Don't dump stack trace to error when not panicking 2022-11-16 13:48:30 +01:00
35fa43f47c Other: Properly handle SMTP to list in send recorder
Checking the BCC header is unreliable; it is usually omitted from messages.
Instead, we can use the SMTP "to" list for deduplication.
2022-11-16 13:48:30 +01:00
4df8ce1b58 Other: Linter to ignore test parameters 2022-11-16 13:48:30 +01:00
83c7396f2d Other: Separate getMessageHash from sendRecorder 2022-11-16 13:48:30 +01:00
709922c383 Other(lint): gofmt 2022-11-16 13:48:30 +01:00
16978b8949 Other(test): Add send recorder test case 2022-11-16 13:48:30 +01:00
7745b68228 GODT-1777: Also include Reply-To and In-Reply-To in message hash 2022-11-16 13:48:30 +01:00
036a416a25 GODT-1777: Message de-duplication in IMAP (+ cleanup) 2022-11-16 13:48:30 +01:00
c9808d07df GODT-1777: Message de-duplication in SMTP 2022-11-16 13:48:30 +01:00
8e34b51c77 GODT-1940: Fix message encryption
Update liteapi to v0.36.0 to include message encryption fix and fix
compile errors related to update.
2022-11-16 13:48:30 +01:00
afc5307a23 Other: Don't close IMAP/SMTP listeners if they could not be created 2022-11-16 13:48:30 +01:00
6fbf6d90dc Other: Fix IMAP/SMTP/Login leaks/race conditions
Depending on the timing of bridge closure, it was possible for the 
IMAP/SMTP servers to not have started serving yet. By grouping this in
a cancelable goroutine group (*xsync.Group), we mitigate this issue.

Further, depending on internet disconnection timing during user login,
it was possible for a user to be improperly logged in. This change 
fixes this and adds test coverage for it.

Lastly, depending on timing, certain background tasks (updates check,
connectivity ping) could be improperly started or closed. This change
groups them in the *xsync.Group as well to be closed properly.
2022-11-16 13:48:30 +01:00
828385b049 Other: Fix user sync leaks/race conditions
This fixes various race conditions and leaks related to the user's sync
and API event stream. It was possible for a sync/stream to begin after a
user was already closed; this change prevents that by managing the
goroutines related to sync/stream within cancellable groups.
2022-11-16 13:48:30 +01:00
6bbaf03f1f Other: Fix goroutine leaks in sync tests
Add missing Close calls.

Properly handle nil channel for `user.startSync`.

This patch also updated liteapi and Gluon to latest master and dev
version respectively.
2022-11-16 13:48:30 +01:00
0f125196a6 Other: Bump go-smtp version to fix race condition
There was a race condition internal to the go-smtp library.
In order to fix it, a version bump was necessary.
However, this significantly changed the library interface.
2022-11-16 13:48:30 +01:00
974735d415 Other: Bump liteapi version to fix goroutine leaks 2022-11-16 13:48:30 +01:00
81f4ef609b Other: Mitigate double-unlock of user keyring
We need to unlock the user keyring anyway to unlock the address keyring,
so we should just return it instead of re-unlocking the user keyring
when sending a message.
2022-11-16 13:48:30 +01:00
80d3f7d179 Other: Set \Draft flag on messages in drafts mailbox 2022-11-16 13:48:30 +01:00
c4343e0124 Other: Bump liteapi and clean up tests a bit 2022-11-16 13:48:30 +01:00
04b6571cb8 Other: Handle Seen/Flagged IMAP flags when APPENDing a message
When an IMAP client appends a message to a mailbox, it can specify
which flags it wants the appended message to have. We need to handle
these in a proton-specific way; not-seen messages need to be imported
with the Unread bool set to true, and flagged messages need to
additionally be imported with the Starred label.
2022-11-16 13:48:30 +01:00
a7a7d9a3d4 GODT-1742: Implement hide All Mail 2022-11-16 12:26:09 +01:00
1729c085c7 Other: Fix user logout hangs due to sync 2022-11-16 12:26:09 +01:00
bf29090ffa Other: Fix log message, rename test fixture 2022-11-16 12:26:09 +01:00
ca132881f9 Other: Update Gluon to v0.13.0 2022-11-16 12:26:09 +01:00
d47b5b99c5 GODT-1813: Cleanup old go-imap cache files 2022-11-16 12:26:09 +01:00