Commit Graph

2500 Commits

Author SHA1 Message Date
c57b52ef23 test(GODT-2746): Integration tests for reporting a problem
- Add function for checking header in form-data request

Bug reports are sent with multipart/form-data, and the function parses this,
then compares the needed field with the wanted
Also, added the step definition.

- Add functions for reporting a bug with changes

Be able to report a bug by changing the value of a single field, or multiple fields through a JSON format

- Add integration tests for reporting a problem
2023-09-01 04:52:42 +00:00
44e103edd5 chore: Umshiang Bridge 3.5.0 changelog. 2023-08-31 16:39:13 +02:00
970de4c205 fix(GODT-2828): Fix negative report time 2023-08-31 11:52:48 +00:00
a189c35899 fix(GODT-2828): Fix sync progress report after restart
Number of already synced messages also needs to be multiplied by the
number of sync stages to have accurate progress reporting.
2023-08-31 11:52:48 +00:00
c9323a40c8 fix(GODT-2867): do not crash on timeout or context cancel. 2023-08-31 11:14:44 +00:00
7a192d50db feat(GODT-2891): Allow message create & delete during sync
Incoming messages which arrive into labels we know during sync are now
presented to the IMAP clients.

We also allow messages to be deleted while syncing if deleted on other
clients.

Other operations such as moving, marking messages as read and label
operations need to be considered in a follow up patch as they are far
more complex.
2023-08-31 10:47:47 +02:00
a5a9bd762d feat(GODT-2848): Decouple IMAP service from Event Loop
The IMAP service no longer blocks the event loop from progressing. When
syncing the event loop continues as usual, allow other parts of the
bridge to continue updating.

Once the sync is complete, it resets the event id back to the last
handled id before sync started and then we play catch up.
2023-08-30 14:10:36 +02:00
dd7db00f74 chore: Umshiang Bridge 3.5.0 changelog. 2023-08-29 14:35:49 +02:00
a6c20f698c feat(GODT-2828): Increase sync progress report frequency
We now report sync progress after a batch completes each stage.
2023-08-29 11:50:50 +00:00
f7252ed40e fix(GODT-2693): Duplicate messages in sent folder
* Only call `RemoveOnFail` on error. It was also called on success.
* Add some logging to track send times.
* Fix non-deterministic hash from `rfc8222.GetMessageHash`.
    https://github.com/ProtonMail/gluon/pull/395
2023-08-29 11:50:50 +00:00
66c30716ea fix(GODT-2867): get attachment returns API error on network problem. 2023-08-29 13:20:45 +02:00
06d7fdf26f fix(GODT-2805): Ignore Contact Group Labels 2023-08-29 07:25:00 +00:00
8aee732fe8 fix(GODT-2866): Add 429/5xx Retry to Event Service 2023-08-29 07:25:00 +00:00
47233752f5 fix(GODT-2855): fix for text overlapping in settings view. 2023-08-28 13:00:33 +00:00
cb6e51531b test: Fix TestBridge_SyncWithOnGoingEvents
Checking for the initial status of the original check as this is not
required to verify the intended behavior. The original check is also
prone to issues related to event processing order.
2023-08-28 13:55:14 +02:00
0b9b886039 fix(GODT-2829): Fix new sync service bugs
* Fix wrong context use for message downloads
* Fix delete of sync data failing due ErrNotFound
* Pre-allocate attachment data buffer before download
* Fix calculation of progress if message count is higher than total
2023-08-28 13:37:55 +02:00
1fa0d77b10 chore: Add trace profiling option 2023-08-28 11:23:41 +02:00
efbe84964f feat(GODT-2829): Integrate new sync service
Update imap service to use the new sync service.

The new sync state is stored as simple file on disk to avoid contention
with concurrent vault writes.
2023-08-25 15:21:00 +02:00
aa77a67a1c fix(GODT-2829): Sync Service fixes
Fix tracking of child jobs. The build stage splits the incoming work
even further, but this was not reflected in the wait group counter. This
also fixes an issue where the cache was cleared to late.

Add more debug info for analysis.

Refactor sync state interface in order to have persistent sync rate.
2023-08-25 15:03:27 +02:00
78f7cbdc79 feat(GODT-2829): New Sync Service
Implementation of the new sync service that interleaves syncing jobs for
all active users.

It also includes improvements to the message downloader. The download
will now auto rate limit the parallel workers based on the server
responses.

Additionally each of the stages is now tested in isolation to ensure the
behavior matches the expectations.

Finally, this patch does not replace the existing IMAP sync. A follow up
patch is necessary to integrate the IMAP bits into the interfaces
required by these changes.
2023-08-23 16:12:19 +02:00
a731237701 test(GODT-2871): tests for new telemetry logic. 2023-08-22 11:15:47 +02:00
f557666b4d feat(GODT-2871): is telemetry enabled as service. 2023-08-21 14:08:47 +02:00
5f89e85139 chore: Under Bridge 3.5.0 changelog. 2023-08-17 12:39:31 +02:00
9fb922d18f chore: merge master to devel 2023-08-17 12:04:17 +02:00
884c6ed932 fix(GODT-2865): add error on failed unlock. 2023-08-17 08:19:25 +00:00
db77bd4983 test(GODT-2873): Wait for Gluon Watcher to finish
Avoids go-routine leak checker entry.

https://github.com/ProtonMail/gluon/pull/393
2023-08-17 06:31:25 +00:00
85bfcf7158 test(GODT-2744): Add integration tests for moving messages (with MOVE support) 2023-08-17 06:22:12 +00:00
09a5f8ac0f test(GODT-2872): Fix nightly job 2023-08-16 12:56:40 +00:00
81f81a63e8 test(GODT-2742): Add more integration tests regarding drafts 2023-08-16 10:56:23 +00:00
e724fafe2f chore: checkout features from trift to devel 2023-08-15 15:40:53 +02:00
cba73997cd fix(GODT-2802): Fix missing key pass input for keyring unlock 2023-08-14 13:09:22 +02:00
29f44fc312 fix(GODT-2814): Server Manager shutdown task cancel 2023-08-14 11:04:53 +02:00
41c125f65e feat(GODT-2803): Gluon IMAP State access
Update to latest Gluon to allow access to the database for bridge. The
cache is placed in a `SharedCache` type to ensure that we respect calls
to `Connector.Close`.

In theory, this should never trigger an error, but this way we can catch
it if it happens.

https://github.com/ProtonMail/gluon/pull/391
2023-08-14 09:51:49 +02:00
0a555bf767 chore: Trift Bridge 3.4.1 changelog. v3.4.1 2023-08-14 08:33:51 +02:00
24331f9715 fix(GODT-2803): Separate conditions to pause event loop for IMAP
Add two separate toggles to control event loop pausing. This is required
to prevent cases where the bridge requests the event loop to be paused
but a sync process completes and resumes the event loop.

For the loop to resume now both states need to be set to false. This
will be removed once GODT-2848 is implemented.
2023-08-11 14:32:18 +02:00
a5500629e5 feat(GODT-2803): Add Event Poll Waiter
Wait for the current event poll to finish publishing events after a
request to pause the event loop. This is required to change the gluon
cache directory.
2023-08-11 14:32:18 +02:00
a46533dcf2 feat(GODT-2787): Force Scrollview to top when re-opening questions set. 2023-08-11 08:43:23 +02:00
12183fbf05 feat(GODT-2787): Tweaking Bug Report form with last Review. 2023-08-11 05:39:34 +00:00
c90248920a ci(GODT-2717): Create a job that will run on schedule 2023-08-10 12:19:59 +00:00
ccb9b7f81c feat(GODT-2787): Fix vertical alignement on CategoryItem. 2023-08-10 08:43:37 +00:00
78c0651661 feat(GODT-2842): Implement Bug Report Fallback notification. 2023-08-10 09:46:12 +02:00
d72980e443 fix(GODT-2859): Fix scope of the function + rename properly. 2023-08-10 08:32:54 +02:00
b24937b666 fix(GODT-2859): Fix lint. 2023-08-10 08:30:46 +02:00
5ca9ec6674 fix(GODT-2859): Trigger user resync while updating from 3.4.0 to 3.4.1. 2023-08-10 08:28:39 +02:00
4ee6da4baa chore(GODT-2848): Simplify User Event Service
Rather than having the services subscribe to each individual event type,
have only one subscriber for the whole event object. This spreads the
logic to the service through the `EventHandler` type.

This is important for the next stage where the IMAP service will be
required to apply events from the past.
2023-08-09 16:11:09 +02:00
e8ce8942be feat(GODT-2788): Add preview to bug report validation. 2023-08-09 11:39:08 +00:00
9d0d3708af feat(GODT-2808): Apply comment from Bug Report content review. 2023-08-09 11:07:15 +00:00
4c908aac7c fix(GODT-2857): Do not check changed values in clear recent flag
https://github.com/ProtonMail/gluon/pull/390
2023-08-09 10:18:30 +02:00
6eb1878f66 fix(GODT-2827): Restore ticker to event poller 2023-08-09 10:17:41 +02:00
826dc2e5c3 test(GODT-2743): Sync high number of messages 2023-08-08 14:37:00 +00:00