diff --git a/doc/bridge.md b/doc/bridge.md deleted file mode 100644 index 969a322f..00000000 --- a/doc/bridge.md +++ /dev/null @@ -1,135 +0,0 @@ -# Bridge - -## Main blocks - -This is basic overview of the main bridge blocks. - -Note connection between IMAP/SMTP and PMAPI. IMAP and SMTP packages are in the queue to be refactored -and we would like to try to have functionality in bridge core or bridge utilities (such as messages) -than direct usage of PMAPI from IMAP or SMTP. Also database (BoltDB) should be moved to bridge core. - -```mermaid -graph LR - S[Server] - C[Client] - U[User] - - subgraph "Bridge app" - Core[Bridge core] - API[PMAPI] - Store - DB[BoltDB] - Frontend["Qt / CLI"] - IMAP - SMTP - - IMAP --> Store - IMAP --> Core - SMTP --> Core - SMTP --> API - Core --> API - Core --> Store - Store --> API - Store --> DB - Frontend --> Core - - end - - C --> IMAP - C --> SMTP - U --> Frontend - API --> S -``` - -## Code structure - -More detailed graph of main types used in Bridge app and connection between them. Here is already -communication to PMAPI only from bridge core which is not true, yet. IMAP and SMTP are still calling -PMAPI directly. - -```mermaid -graph TD - - C["Client (e.g. Thunderbird)"] - PM[Proton Mail Server] - - subgraph "Bridge app" - subgraph "Bridge core" - B[Bridge] - U[User] - - B --> U - end - - subgraph Store - StoreU[Store User] - StoreA[Address] - StoreM[Mailbox] - - StoreU --> StoreA - StoreA --> StoreM - end - - subgraph Credentials - CredStore[Store] - Creds[Credentials] - - CredStore --> Creds - end - - subgraph Frontend - CLI - Qt - end - - subgraph IMAP - IB[IMAP backend] - IA[IMAP address] - IM[IMAP mailbox] - - IB --> B - IB --> IA - IA --> IM - IA --> U - IA --> StoreA - IM --> StoreM - end - - subgraph SMTP - SB[SMTP backend] - SS[SMTP session] - - SB --> B - SB --> SS - SS --> U - end - end - - subgraph PMAPI - AC[Client] - end - - C --> IB - C --> SB - - CLI --> B - Qt --> B - - U --> CredStore - U --> Creds - - U --> StoreU - - StoreU --> AC - StoreA --> AC - StoreM --> AC - - B --> AC - U --> AC - - AC --> PM -``` - -## How to debug - -Run `make run-debug` which starts [Delve](https://github.com/go-delve/delve). diff --git a/doc/communication.md b/doc/communication.md deleted file mode 100644 index 6724d0a5..00000000 --- a/doc/communication.md +++ /dev/null @@ -1,114 +0,0 @@ -# Communication - -## First login and sync - -When user logs in to the bridge for the first time, immediately starts the first sync. -First sync downloads all headers of all e-mails and creates database to have proper UIDs -and indexes for IMAP. See [database](database.md) for more information. - -By default, whenever it's possible, sync downloads only all e-mails maiblox which already -have list of labels so we can construct all mailboxes (inbox, sent, trash, custom folders -and labels) without need to download each e-mail headers many times. - -Note that we need to download also bodies to calculate size of the e-mail and set proper -content type (clients uses content type for guess if e-mail contains attachment)--but only -body, not attachment. Also it's downloaded only for the first time. After that we store -those information in our database so next time we only sync headers, labels and so on. - -First sync takes some time. List of 150 messages takes about second and then we need to -download bodies for each message. We still need to do some optimalizations. Anyway, if -user has reasonable amount of e-mails, there is good chance user will see e-mails in the -client right after adding account. - -When account is added to client, client start the sync. This sync will ask Bridge app -for all headers (done quickly) and then starts to download all bodies and attachment. -Unfortunately for some e-mail more than once if the same e-mail is in more mailboxes -(e.g. inbox and all mail)--there is no way to tell over IMAP it's the same message. - -After successful login of client to IMAP, Bridge starts event loop. That periodicly ask -servers (each 30 seconds) for new updates (new message, keys, …). - -```mermaid -sequenceDiagram - participant S as Server - participant B as Bridge - participant C as Client - - Note right of B: Set up PM account
by user - - loop First sync - B ->> S: Fetch body and attachments - Note right of B: Build local database
(e-mail UIDs) - end - - Note right of C: Set up IMAP/SMTP
by user - - C ->> B: IMAP login - B ->> S: Authenticate user - Note right of B: Create IMAP user - - loop Event loop, every 30 sec - B ->> S: Fetch e-mail headers - B ->> C: Send IMAP IDLE response - end - - C ->> B: IMAP LIST directories - - loop Client sync - C ->> B: IMAP SELECT directory - C ->> B: IMAP SEARCH e-mails UIDs - C ->> B: IMAP FETCH of e-mail UID - B ->> S: Fetch body and attachments - Note right of B: Decrypt message
and attachment - B ->> C: IMAP response - end -``` - -## IMAP IDLE extension - -IMAP IDLE is extension, it has to be supported by both client and server. IMAP server (in our case -the bridge) supports it so clients can use it. It works by issuing `IDLE` command by the client and -keeps the connection open. When the server has some update, server (the bridge) will respond to that -by `EXISTS` (new message), `APPEND` (imported message), `EXPUNGE` (deleted message) or `MOVE` response. - -Even when there is connection with IDLE open, server can mark the client as inactive. Therefore, -it's recommended the client should reissue the connection after each 29 minutes. This is not the -real push and can fail! - -Our event loop is also simple pull and it will trigger IMAP IDLE when we get some new update from -the server. Would be good to have push from the server, but we need to wait for the support on API. - -RFC: https://tools.ietf.org/html/rfc2177 - -```mermaid -sequenceDiagram - participant S as Server - participant B as Bridge - participant C as Client - - C ->> B: IMAP IDLE - - loop Every 30 seconds - S ->> B: Checking events - B ->> C: IMAP response - end -``` - -## Sending e-mails - -E-mail are sent over standard SMTP protocol. Our bridge takes the message, encrypts and sent it -further to our server which will then send the message to its final destination. The important -and tricky part is encryption. See [encryption](encryption.md) or [PMEL document](https://docs.google.com/document/d/1lEBkG0DC5FOWlumInKtu4a9Cc1Eszp48ZhFy9UpPQso/edit) -for more information. - -```mermaid -sequenceDiagram - participant S as Server - participant B as Bridge - participant C as Client - - C ->> B: SMTP send e-mail - Note right of B: Encrypt messages - B ->> S: Send encrypted e-mail - B ->> C: Respond OK -``` diff --git a/doc/database.md b/doc/database.md deleted file mode 100644 index 33a01b0d..00000000 --- a/doc/database.md +++ /dev/null @@ -1,27 +0,0 @@ -# Database - -Bridge needs to have a small database to pair our IDs with IMAP UIDs and indexes. IMAP protocol -requires every message to have an unique UID in mailbox. In this context, mailbox is not an account, -but a folder or label. This means that one message can have more UIDs, one for each mailbox (folder), -and that two messages can have the same UID, but each for different mailbox (folder). - -IMAP index is just an index. Look at it like to an array: `["UID1", "UID2", "UID3"]`. We can access -message by UID or index; for example index 2 and UID `UID2`. When this message is deleted, we need -to re-index all following messages. The array will look now like `["UID1", "UID3"]` and the last -message can be accessed by index 2 or UID `UID3`. - -See RFCs for more information: - -* https://tools.ietf.org/html/rfc822 -* https://tools.ietf.org/html/rfc3501 - -Our database is currently built on BBolt and have those buckets (key-value storage): - -* Message metadata bucket: - - * `[metadataBucket][API_ID] -> pmapi.Message{subject, from, to, size, other headers...}` (without body or attachment) - -* Mapping buckets - - * `[mailboxesBucket][addressID-mailboxID][api_ids][API_ID] -> UID` - * `[mailboxesBucket][addressID-mailboxID][imap_ids][UID] -> API_ID` diff --git a/doc/encryption.md b/doc/encryption.md deleted file mode 100644 index 442a5b50..00000000 --- a/doc/encryption.md +++ /dev/null @@ -1,12 +0,0 @@ -# Encryption - -Encryption is done in PMAPI, bridge utils and bridge itself. The best would be to keep encryption -in PMAPI and bridge utils (in package such as messages). All packages are using our high-level -GopenPGP library on top of OpenPGP. - -## `gopenpgp.KeyRing` - -We use one `KeyRing` per address. Our usage then contains all keys for specific address. Primary -key is always on the first position, then there old ones to be able to decrypt last e-mail. -OpenPGP encrypts given message with all available keys, so we need to first get first (primary) -key for encryption to have message encrypted only once with primary key. diff --git a/doc/index.md b/doc/index.md deleted file mode 100644 index d46f2cb4..00000000 --- a/doc/index.md +++ /dev/null @@ -1,9 +0,0 @@ -# Bridge Documentation - -Documentation pages in order to read for a novice: - -* [Bridge code](bridge.md) -* [Internal Bridge database](database.md) -* [Communication between Bridge, Client and Server](communication.md) -* [Encryption](encryption.md) - diff --git a/doc/updates.md b/doc/updates.md deleted file mode 100644 index 53ffe984..00000000 --- a/doc/updates.md +++ /dev/null @@ -1,103 +0,0 @@ -# Update mechanism of Bridge - -There are multiple options how to change version of application: -* Automatic in-app update -* Manual in-app update -* Manual install - -In-app update ends with restarting bridge into new version. Automatic in-app -update is downloading, verifying and installing the new version immediately -without user confirmation. For manual in-app update user needs to confirm first. -Update is done from special update file published on website. - -The manual installation requires user to download, verify and install manually -using installer for given OS. - -The bridge is installed and executed differently for given OS: - -* Windows and Linux apps are using launcher mechanism: - * There is system protected installation path which is created on first - install. It contains bridge exe and launcher exe. When users starts - bridge the launcher is executed first. It will check update path compare - version with installed one. The newer version then is then executed. - * Update mechanism means to replace files in update folder which is located - in user space. - -* macOS app does not use launcher - * No launcher, only one executable - * In-App update replaces the bridge files in installation path directly - - -```mermaid -flowchart LR - subgraph Frontend - U[User requests
version check] - ManIns((Notify user about
manual install
is needed)) - R((Notify user
about restart)) - ManUp((Notify user about
manual update)) - NF((Notify user about
force update)) - - ManUp -->|Install| InstFront[Install] - InstFront -->|Ok| R - InstFront -->|Error| ManIns - - U --> CheckFront[Check online] - CheckFront -->|Ok| IAFront{Is new version
and applicable?} - CheckFront -->|Error| ManIns - - IAFront -->|No| Latest((Notify user
has latest version)) - IAFront -->|Yes| CanInstall{Can update?} - CanInstall -->|No| ManIns - CanInstall -->|Yes| NotifOrInstall{Is automatic
update enabled?} - NotifOrInstall -->|Manual| ManUp - end - - - subgraph Backend - W[Wait for next check] - - W --> Check[Check online] - - Check --> NV{Has new
version?} - Check -->|Error| W - NV -->|No new version| W - IA{Is install
applicable?} - NV -->|New version
available| IA - IA -->|Local rollout
not enough| W - IA -->|Yes| AU{Is automatic\nupdate enabled?} - - AU -->|Yes| CanUp{Can update?} - CanUp -->|No| ManIns - - CanUp -->|Yes| Ins[Install] - Ins -->|Error| ManIns - Ins -->|Ok| R - - AU -->|No| ManUp - ManUp -->|Ignore| W - - - F[Force update] - F --> NF - end - - ManIns --> Web[Open web page] - NF --> Web - ManUp --> Web - R --> Re[Restart] - NF --> Q[Quit bridge] - NotifOrInstall -->|Automatic| W -``` - - -The non-trivial is to combine the update with setting change: -* turn off/on automatic in-app updates -* change from stable to beta or back - -_TODO fill flow chart details_ - - -We are not support downgrade functionality. Only some circumstances can lead to -downgrading the app version. - -_TODO fill flow chart details_