437 Commits

Author SHA1 Message Date
d3f8297eb4 fix(GODT-3013): IMAP service getting "stuck"
* Ensure IMAP service sync cancel request waits until the sync has
  completely cancelled rather than just signaling. It's possible that
  due the context reset on `group.Cancel` that something may have not
  have been bookmarked correctly in subsequent sync restarts.

* Handle connection lost/restored events in the services. Removes the
  need to lock bridge users. Which could conflict with other ongoing
  lock operations. Additionally, it ensure that if one service is
  blocked it doesn't block the entire bridge.

* Revise access to bridge user locks.
2023-10-11 11:20:53 +01:00
ab9a758d63 fix(GODT-3003): Ensure IMAP State is reset after vault corruption
After we detect that the user has suffered the GODT-3003 bug due the
vault corruption not ensuring that a previous sync state would be
erased, we patch the gluon db directly and then reset the sync state.

After the account is added, the sync is automatically triggered and the
account state fixes itself.
2023-10-09 10:23:58 +01:00
de0b6c0737 feat(GODT-2835): Bump GPA adding support for AsyncAttachments for BugReport +... 2023-10-03 13:43:16 +00:00
3bf4282037 feat(GODT-2940): allow 3 attempts for mailbox password. 2023-10-02 16:50:07 +02:00
48d1ca1e72 fix(GODT-2989): allow to send bug report when no account connected. 2023-10-02 13:34:40 +00:00
7a192d50db feat(GODT-2891): Allow message create & delete during sync
Incoming messages which arrive into labels we know during sync are now
presented to the IMAP clients.

We also allow messages to be deleted while syncing if deleted on other
clients.

Other operations such as moving, marking messages as read and label
operations need to be considered in a follow up patch as they are far
more complex.
2023-08-31 10:47:47 +02:00
a5a9bd762d feat(GODT-2848): Decouple IMAP service from Event Loop
The IMAP service no longer blocks the event loop from progressing. When
syncing the event loop continues as usual, allow other parts of the
bridge to continue updating.

Once the sync is complete, it resets the event id back to the last
handled id before sync started and then we play catch up.
2023-08-30 14:10:36 +02:00
cb6e51531b test: Fix TestBridge_SyncWithOnGoingEvents
Checking for the initial status of the original check as this is not
required to verify the intended behavior. The original check is also
prone to issues related to event processing order.
2023-08-28 13:55:14 +02:00
efbe84964f feat(GODT-2829): Integrate new sync service
Update imap service to use the new sync service.

The new sync state is stored as simple file on disk to avoid contention
with concurrent vault writes.
2023-08-25 15:21:00 +02:00
a731237701 test(GODT-2871): tests for new telemetry logic. 2023-08-22 11:15:47 +02:00
41c125f65e feat(GODT-2803): Gluon IMAP State access
Update to latest Gluon to allow access to the database for bridge. The
cache is placed in a `SharedCache` type to ensure that we respect calls
to `Connector.Close`.

In theory, this should never trigger an error, but this way we can catch
it if it happens.

https://github.com/ProtonMail/gluon/pull/391
2023-08-14 09:51:49 +02:00
6eb1878f66 fix(GODT-2827): Restore ticker to event poller 2023-08-09 10:17:41 +02:00
20b188368a feat(GODT-2799): SMTP service interacts directly with Server Manager
Bridge no longer needs to manually add and remove accounts from the
service.
2023-08-08 14:14:52 +02:00
ded4f370dc fix(GODT-2759): Use examine rather than select for fetching
When fetching messages in the debug mailbox state command, use read only
mode to avoid modifying the mailbox state.
2023-08-08 13:05:39 +02:00
9a96588afb feat(GODT-2814): Standalone Server Manager
Convert ServerManger into a standalone service so that it can become a
self contained module.
2023-08-07 16:47:41 +02:00
80d729e3e5 feat(GODT-2816): Wait until mandatory fields are filled then fill body and title. 2023-08-01 14:49:53 +00:00
716de7f45a test: Fix event registration in TestBridge_SyncWithOngoingEvents
We need to register the event subscriber earlier to avoid missing the
event later.
2023-07-31 12:36:06 +02:00
750de0cd31 test: TestBridge_SendAddTextBodyPartIfNotExists eventually fix
It may take a couple of fetch requests until the pending state updates
get applied.
2023-07-31 12:36:06 +02:00
823ca4d207 feat(GODT-2822): Integrate and activate all service
The bridge now runs on the new architecture.
2023-07-31 12:36:03 +02:00
a187747c7c feat(GODT-2802): IMAP Serivce
Handles all IMAP related tasks. Unlike the previous iteration, this new
service automatically adds and removes users from Gluon by interfacing
with server manager.
2023-07-31 11:06:47 +02:00
90c34406ba test: Fix deadlock in chToType
Make sure the go routine is cancelled if there is a new event, but no
one is reading it after a call to done.
2023-07-28 13:32:28 +02:00
bc66841cdc chore(GODT-2799): Move SMTP backend to SMTP service module 2023-07-19 09:12:33 +02:00
8d028966c7 chore(GODT-2799): Move Identifier interface to separate module
Required so we can move the smtp backend into the smtp service module.
2023-07-19 09:07:02 +02:00
d120bbeffc chore(GODT-2799): Separate account states for SMTP Backend
Rather than accessing the Bridge user list, each user register their
individual SMTP service with the server manager.

Note that some dependencies on the user data are hidden behind the
`UserInterface`. These will be removed in a future patch.
2023-07-19 09:07:02 +02:00
66bc911652 chore: fix typos found by codespell 2023-07-07 15:31:14 +02:00
0d57e3645a test: Add require.Eventually to TestBridge_UserAgentFromSMTPClient 2023-07-06 14:40:17 +02:00
908ed3e723 feat(GODT-2759): Check for oprhan messages
Orphan messages are messages which are only in AllMail, AllDrafts or
AllSent.
2023-07-06 14:02:05 +02:00
7411073c08 feat(GODT-2759): Add prompt to download missing messages for analysis
This will download the missing messages into a temporary directory and
decrypt them along with the metadata so we can attempt analyze them once
submitted to see what is going wrong.
2023-07-06 14:02:05 +02:00
7d838375bb feat(GODT-2759): CLI debug commands
Add debug commands to CLI to diagnose potential bride problems.
Currently we only have a command which validates whether the state of
all the mailboxes reported by IMAP matches what is currently available
on the proton servers.
2023-07-06 14:02:05 +02:00
f545f30ec0 fix(GODT-2774): only check telemetry availability for the current user. 2023-07-06 13:38:06 +02:00
40c48ba804 fix(GODT-2774): Add external context to telemetry tasks
This ensures they get cancelled if the parent context becomes invalid
2023-07-06 13:09:35 +02:00
eaa673c4e4 fix(GODT-2708): fix dimensions event format + handling of ReportClicked event. 2023-07-04 13:04:30 +02:00
cc17366c1c fix(GODT-2578): Refresh literals appended to Sent folder
Whenever a message gets moved to the sent folder we should retrieve the
new literal in order to guarantee that, if another client modifies and
sends the message, we always see the latest version of the message and
not a previous state stored in the Gluon cache.

Includes the following Gluon MRs:
* https://github.com/ProtonMail/gluon/pull/374
* https://github.com/ProtonMail/gluon/pull/376

Includes the followin gpa MR:
https://github.com/ProtonMail/go-proton-api/pull/88
2023-07-04 10:39:07 +02:00
234554b459 feat(GODT-2709): Remove the config status file when user is removed. 2023-07-01 07:21:27 +02:00
238929c3ec feat(GODT-2712): Feed config_status with user action while pending. 2023-06-30 09:43:26 +00:00
9482bea8af feat(GODT-2714): Apply PR comments. 2023-06-30 09:43:26 +00:00
a55572e5b3 feat(GODT-2714): Set Configuration Status to Failure and send Recovery event when issue is solved. 2023-06-30 09:43:26 +00:00
68334e3bb8 feat(GODT-2711): Send config_abort event on User removal. 2023-06-30 09:43:26 +00:00
124231c3c7 feat(GODT-2710): Send config success on IMAP/SMTP connection.. 2023-06-30 09:43:26 +00:00
ff11d20d9c feat(GODT-2709): Init Configuration status. 2023-06-30 09:43:26 +00:00
a2c2710760 feat(GODT-2685): update to bug report log attachment logic. 2023-06-19 16:30:36 +02:00
ac00ef1b64 feat(GODT-2666): feat(GODT-2667): introduce sessionID in bridge. 2023-06-07 09:00:33 +02:00
7f7e360cd7 feat(GODT-2673): Use NoClient as UserAgent without any client connected and... 2023-06-01 15:27:56 +00:00
0cef181432 fix(GODT-2437): Fix lint. 2023-05-25 09:25:25 +02:00
262c4c5d95 fix(GODT-2437): Fix test + bump Gluon with silenced report. 2023-05-25 09:12:15 +02:00
bc5de2b884 fix(GODT-2638): Validate messages before import.
Ensure messages have required header fields and adhere to the rfc5322
spec before import.
2023-05-24 06:38:13 +00:00
9c25f56fe6 test: fix flaky tests. 2023-05-22 11:16:56 +00:00
35f0e081a5 fix(GODT-2628): Attempt to fix closed channel panic on logout
It should not be possible to reach this state on purpose. But due to
scheduling and synchronization variances, it is possible in theory that
a UserDeathEvent can occur at the same time as the bridge is closing,
causing a call to `User.Close()` to be executed 2 times.

To avoid this in the future we just clear the map after all the channels
have been closed.
2023-05-17 11:54:02 +00:00
0ef9c9c9c9 fix(GODT-2606): Improve Vault concurrency scopes
Rewrite the vault to have one RWlock rather then two separate locks for
data and reference counts.

In certain circumstance, it could be possible that that different
requests could end up in undefined states if a user got deleted
successfully while at he same time another goroutine/thread is loading
the given user.

While I have not been able to reproduce this in a test, restricting the
access scope to one lock rather than two, should avoid corner cases
where logic code is executing outside of the lock scope.
2023-05-15 13:28:00 +02:00
faf3780eee test: Fix TestBridge_Report 2023-05-12 15:37:33 +02:00