mirror of
https://github.com/ProtonMail/proton-bridge.git
synced 2025-12-13 06:06:44 +00:00
fix(BRIDGE-138): remove deprecated doc.
This commit is contained in:
Xavier Michelon
135
doc/bridge.md
135
doc/bridge.md
@ -1,135 +0,0 @@
|
|||||||
# Bridge
|
|
||||||
|
|
||||||
## Main blocks
|
|
||||||
|
|
||||||
This is basic overview of the main bridge blocks.
|
|
||||||
|
|
||||||
Note connection between IMAP/SMTP and PMAPI. IMAP and SMTP packages are in the queue to be refactored
|
|
||||||
and we would like to try to have functionality in bridge core or bridge utilities (such as messages)
|
|
||||||
than direct usage of PMAPI from IMAP or SMTP. Also database (BoltDB) should be moved to bridge core.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph LR
|
|
||||||
S[Server]
|
|
||||||
C[Client]
|
|
||||||
U[User]
|
|
||||||
|
|
||||||
subgraph "Bridge app"
|
|
||||||
Core[Bridge core]
|
|
||||||
API[PMAPI]
|
|
||||||
Store
|
|
||||||
DB[BoltDB]
|
|
||||||
Frontend["Qt / CLI"]
|
|
||||||
IMAP
|
|
||||||
SMTP
|
|
||||||
|
|
||||||
IMAP --> Store
|
|
||||||
IMAP --> Core
|
|
||||||
SMTP --> Core
|
|
||||||
SMTP --> API
|
|
||||||
Core --> API
|
|
||||||
Core --> Store
|
|
||||||
Store --> API
|
|
||||||
Store --> DB
|
|
||||||
Frontend --> Core
|
|
||||||
|
|
||||||
end
|
|
||||||
|
|
||||||
C --> IMAP
|
|
||||||
C --> SMTP
|
|
||||||
U --> Frontend
|
|
||||||
API --> S
|
|
||||||
```
|
|
||||||
|
|
||||||
## Code structure
|
|
||||||
|
|
||||||
More detailed graph of main types used in Bridge app and connection between them. Here is already
|
|
||||||
communication to PMAPI only from bridge core which is not true, yet. IMAP and SMTP are still calling
|
|
||||||
PMAPI directly.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph TD
|
|
||||||
|
|
||||||
C["Client (e.g. Thunderbird)"]
|
|
||||||
PM[Proton Mail Server]
|
|
||||||
|
|
||||||
subgraph "Bridge app"
|
|
||||||
subgraph "Bridge core"
|
|
||||||
B[Bridge]
|
|
||||||
U[User]
|
|
||||||
|
|
||||||
B --> U
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph Store
|
|
||||||
StoreU[Store User]
|
|
||||||
StoreA[Address]
|
|
||||||
StoreM[Mailbox]
|
|
||||||
|
|
||||||
StoreU --> StoreA
|
|
||||||
StoreA --> StoreM
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph Credentials
|
|
||||||
CredStore[Store]
|
|
||||||
Creds[Credentials]
|
|
||||||
|
|
||||||
CredStore --> Creds
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph Frontend
|
|
||||||
CLI
|
|
||||||
Qt
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph IMAP
|
|
||||||
IB[IMAP backend]
|
|
||||||
IA[IMAP address]
|
|
||||||
IM[IMAP mailbox]
|
|
||||||
|
|
||||||
IB --> B
|
|
||||||
IB --> IA
|
|
||||||
IA --> IM
|
|
||||||
IA --> U
|
|
||||||
IA --> StoreA
|
|
||||||
IM --> StoreM
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph SMTP
|
|
||||||
SB[SMTP backend]
|
|
||||||
SS[SMTP session]
|
|
||||||
|
|
||||||
SB --> B
|
|
||||||
SB --> SS
|
|
||||||
SS --> U
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph PMAPI
|
|
||||||
AC[Client]
|
|
||||||
end
|
|
||||||
|
|
||||||
C --> IB
|
|
||||||
C --> SB
|
|
||||||
|
|
||||||
CLI --> B
|
|
||||||
Qt --> B
|
|
||||||
|
|
||||||
U --> CredStore
|
|
||||||
U --> Creds
|
|
||||||
|
|
||||||
U --> StoreU
|
|
||||||
|
|
||||||
StoreU --> AC
|
|
||||||
StoreA --> AC
|
|
||||||
StoreM --> AC
|
|
||||||
|
|
||||||
B --> AC
|
|
||||||
U --> AC
|
|
||||||
|
|
||||||
AC --> PM
|
|
||||||
```
|
|
||||||
|
|
||||||
## How to debug
|
|
||||||
|
|
||||||
Run `make run-debug` which starts [Delve](https://github.com/go-delve/delve).
|
|
||||||
@ -1,114 +0,0 @@
|
|||||||
# Communication
|
|
||||||
|
|
||||||
## First login and sync
|
|
||||||
|
|
||||||
When user logs in to the bridge for the first time, immediately starts the first sync.
|
|
||||||
First sync downloads all headers of all e-mails and creates database to have proper UIDs
|
|
||||||
and indexes for IMAP. See [database](database.md) for more information.
|
|
||||||
|
|
||||||
By default, whenever it's possible, sync downloads only all e-mails maiblox which already
|
|
||||||
have list of labels so we can construct all mailboxes (inbox, sent, trash, custom folders
|
|
||||||
and labels) without need to download each e-mail headers many times.
|
|
||||||
|
|
||||||
Note that we need to download also bodies to calculate size of the e-mail and set proper
|
|
||||||
content type (clients uses content type for guess if e-mail contains attachment)--but only
|
|
||||||
body, not attachment. Also it's downloaded only for the first time. After that we store
|
|
||||||
those information in our database so next time we only sync headers, labels and so on.
|
|
||||||
|
|
||||||
First sync takes some time. List of 150 messages takes about second and then we need to
|
|
||||||
download bodies for each message. We still need to do some optimalizations. Anyway, if
|
|
||||||
user has reasonable amount of e-mails, there is good chance user will see e-mails in the
|
|
||||||
client right after adding account.
|
|
||||||
|
|
||||||
When account is added to client, client start the sync. This sync will ask Bridge app
|
|
||||||
for all headers (done quickly) and then starts to download all bodies and attachment.
|
|
||||||
Unfortunately for some e-mail more than once if the same e-mail is in more mailboxes
|
|
||||||
(e.g. inbox and all mail)--there is no way to tell over IMAP it's the same message.
|
|
||||||
|
|
||||||
After successful login of client to IMAP, Bridge starts event loop. That periodicly ask
|
|
||||||
servers (each 30 seconds) for new updates (new message, keys, …).
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
sequenceDiagram
|
|
||||||
participant S as Server
|
|
||||||
participant B as Bridge
|
|
||||||
participant C as Client
|
|
||||||
|
|
||||||
Note right of B: Set up PM account<br/>by user
|
|
||||||
|
|
||||||
loop First sync
|
|
||||||
B ->> S: Fetch body and attachments
|
|
||||||
Note right of B: Build local database<br/>(e-mail UIDs)
|
|
||||||
end
|
|
||||||
|
|
||||||
Note right of C: Set up IMAP/SMTP<br/>by user
|
|
||||||
|
|
||||||
C ->> B: IMAP login
|
|
||||||
B ->> S: Authenticate user
|
|
||||||
Note right of B: Create IMAP user
|
|
||||||
|
|
||||||
loop Event loop, every 30 sec
|
|
||||||
B ->> S: Fetch e-mail headers
|
|
||||||
B ->> C: Send IMAP IDLE response
|
|
||||||
end
|
|
||||||
|
|
||||||
C ->> B: IMAP LIST directories
|
|
||||||
|
|
||||||
loop Client sync
|
|
||||||
C ->> B: IMAP SELECT directory
|
|
||||||
C ->> B: IMAP SEARCH e-mails UIDs
|
|
||||||
C ->> B: IMAP FETCH of e-mail UID
|
|
||||||
B ->> S: Fetch body and attachments
|
|
||||||
Note right of B: Decrypt message<br/>and attachment
|
|
||||||
B ->> C: IMAP response
|
|
||||||
end
|
|
||||||
```
|
|
||||||
|
|
||||||
## IMAP IDLE extension
|
|
||||||
|
|
||||||
IMAP IDLE is extension, it has to be supported by both client and server. IMAP server (in our case
|
|
||||||
the bridge) supports it so clients can use it. It works by issuing `IDLE` command by the client and
|
|
||||||
keeps the connection open. When the server has some update, server (the bridge) will respond to that
|
|
||||||
by `EXISTS` (new message), `APPEND` (imported message), `EXPUNGE` (deleted message) or `MOVE` response.
|
|
||||||
|
|
||||||
Even when there is connection with IDLE open, server can mark the client as inactive. Therefore,
|
|
||||||
it's recommended the client should reissue the connection after each 29 minutes. This is not the
|
|
||||||
real push and can fail!
|
|
||||||
|
|
||||||
Our event loop is also simple pull and it will trigger IMAP IDLE when we get some new update from
|
|
||||||
the server. Would be good to have push from the server, but we need to wait for the support on API.
|
|
||||||
|
|
||||||
RFC: https://tools.ietf.org/html/rfc2177
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
sequenceDiagram
|
|
||||||
participant S as Server
|
|
||||||
participant B as Bridge
|
|
||||||
participant C as Client
|
|
||||||
|
|
||||||
C ->> B: IMAP IDLE
|
|
||||||
|
|
||||||
loop Every 30 seconds
|
|
||||||
S ->> B: Checking events
|
|
||||||
B ->> C: IMAP response
|
|
||||||
end
|
|
||||||
```
|
|
||||||
|
|
||||||
## Sending e-mails
|
|
||||||
|
|
||||||
E-mail are sent over standard SMTP protocol. Our bridge takes the message, encrypts and sent it
|
|
||||||
further to our server which will then send the message to its final destination. The important
|
|
||||||
and tricky part is encryption. See [encryption](encryption.md) or [PMEL document](https://docs.google.com/document/d/1lEBkG0DC5FOWlumInKtu4a9Cc1Eszp48ZhFy9UpPQso/edit)
|
|
||||||
for more information.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
sequenceDiagram
|
|
||||||
participant S as Server
|
|
||||||
participant B as Bridge
|
|
||||||
participant C as Client
|
|
||||||
|
|
||||||
C ->> B: SMTP send e-mail
|
|
||||||
Note right of B: Encrypt messages
|
|
||||||
B ->> S: Send encrypted e-mail
|
|
||||||
B ->> C: Respond OK
|
|
||||||
```
|
|
||||||
@ -1,27 +0,0 @@
|
|||||||
# Database
|
|
||||||
|
|
||||||
Bridge needs to have a small database to pair our IDs with IMAP UIDs and indexes. IMAP protocol
|
|
||||||
requires every message to have an unique UID in mailbox. In this context, mailbox is not an account,
|
|
||||||
but a folder or label. This means that one message can have more UIDs, one for each mailbox (folder),
|
|
||||||
and that two messages can have the same UID, but each for different mailbox (folder).
|
|
||||||
|
|
||||||
IMAP index is just an index. Look at it like to an array: `["UID1", "UID2", "UID3"]`. We can access
|
|
||||||
message by UID or index; for example index 2 and UID `UID2`. When this message is deleted, we need
|
|
||||||
to re-index all following messages. The array will look now like `["UID1", "UID3"]` and the last
|
|
||||||
message can be accessed by index 2 or UID `UID3`.
|
|
||||||
|
|
||||||
See RFCs for more information:
|
|
||||||
|
|
||||||
* https://tools.ietf.org/html/rfc822
|
|
||||||
* https://tools.ietf.org/html/rfc3501
|
|
||||||
|
|
||||||
Our database is currently built on BBolt and have those buckets (key-value storage):
|
|
||||||
|
|
||||||
* Message metadata bucket:
|
|
||||||
|
|
||||||
* `[metadataBucket][API_ID] -> pmapi.Message{subject, from, to, size, other headers...}` (without body or attachment)
|
|
||||||
|
|
||||||
* Mapping buckets
|
|
||||||
|
|
||||||
* `[mailboxesBucket][addressID-mailboxID][api_ids][API_ID] -> UID`
|
|
||||||
* `[mailboxesBucket][addressID-mailboxID][imap_ids][UID] -> API_ID`
|
|
||||||
@ -1,12 +0,0 @@
|
|||||||
# Encryption
|
|
||||||
|
|
||||||
Encryption is done in PMAPI, bridge utils and bridge itself. The best would be to keep encryption
|
|
||||||
in PMAPI and bridge utils (in package such as messages). All packages are using our high-level
|
|
||||||
GopenPGP library on top of OpenPGP.
|
|
||||||
|
|
||||||
## `gopenpgp.KeyRing`
|
|
||||||
|
|
||||||
We use one `KeyRing` per address. Our usage then contains all keys for specific address. Primary
|
|
||||||
key is always on the first position, then there old ones to be able to decrypt last e-mail.
|
|
||||||
OpenPGP encrypts given message with all available keys, so we need to first get first (primary)
|
|
||||||
key for encryption to have message encrypted only once with primary key.
|
|
||||||
@ -1,9 +0,0 @@
|
|||||||
# Bridge Documentation
|
|
||||||
|
|
||||||
Documentation pages in order to read for a novice:
|
|
||||||
|
|
||||||
* [Bridge code](bridge.md)
|
|
||||||
* [Internal Bridge database](database.md)
|
|
||||||
* [Communication between Bridge, Client and Server](communication.md)
|
|
||||||
* [Encryption](encryption.md)
|
|
||||||
|
|
||||||
103
doc/updates.md
103
doc/updates.md
@ -1,103 +0,0 @@
|
|||||||
# Update mechanism of Bridge
|
|
||||||
|
|
||||||
There are multiple options how to change version of application:
|
|
||||||
* Automatic in-app update
|
|
||||||
* Manual in-app update
|
|
||||||
* Manual install
|
|
||||||
|
|
||||||
In-app update ends with restarting bridge into new version. Automatic in-app
|
|
||||||
update is downloading, verifying and installing the new version immediately
|
|
||||||
without user confirmation. For manual in-app update user needs to confirm first.
|
|
||||||
Update is done from special update file published on website.
|
|
||||||
|
|
||||||
The manual installation requires user to download, verify and install manually
|
|
||||||
using installer for given OS.
|
|
||||||
|
|
||||||
The bridge is installed and executed differently for given OS:
|
|
||||||
|
|
||||||
* Windows and Linux apps are using launcher mechanism:
|
|
||||||
* There is system protected installation path which is created on first
|
|
||||||
install. It contains bridge exe and launcher exe. When users starts
|
|
||||||
bridge the launcher is executed first. It will check update path compare
|
|
||||||
version with installed one. The newer version then is then executed.
|
|
||||||
* Update mechanism means to replace files in update folder which is located
|
|
||||||
in user space.
|
|
||||||
|
|
||||||
* macOS app does not use launcher
|
|
||||||
* No launcher, only one executable
|
|
||||||
* In-App update replaces the bridge files in installation path directly
|
|
||||||
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart LR
|
|
||||||
subgraph Frontend
|
|
||||||
U[User requests<br>version check]
|
|
||||||
ManIns((Notify user about<br>manual install<br>is needed))
|
|
||||||
R((Notify user<br>about restart))
|
|
||||||
ManUp((Notify user about<br>manual update))
|
|
||||||
NF((Notify user about<br>force update))
|
|
||||||
|
|
||||||
ManUp -->|Install| InstFront[Install]
|
|
||||||
InstFront -->|Ok| R
|
|
||||||
InstFront -->|Error| ManIns
|
|
||||||
|
|
||||||
U --> CheckFront[Check online]
|
|
||||||
CheckFront -->|Ok| IAFront{Is new version<br>and applicable?}
|
|
||||||
CheckFront -->|Error| ManIns
|
|
||||||
|
|
||||||
IAFront -->|No| Latest((Notify user<br>has latest version))
|
|
||||||
IAFront -->|Yes| CanInstall{Can update?}
|
|
||||||
CanInstall -->|No| ManIns
|
|
||||||
CanInstall -->|Yes| NotifOrInstall{Is automatic<br>update enabled?}
|
|
||||||
NotifOrInstall -->|Manual| ManUp
|
|
||||||
end
|
|
||||||
|
|
||||||
|
|
||||||
subgraph Backend
|
|
||||||
W[Wait for next check]
|
|
||||||
|
|
||||||
W --> Check[Check online]
|
|
||||||
|
|
||||||
Check --> NV{Has new<br>version?}
|
|
||||||
Check -->|Error| W
|
|
||||||
NV -->|No new version| W
|
|
||||||
IA{Is install<br>applicable?}
|
|
||||||
NV -->|New version<br>available| IA
|
|
||||||
IA -->|Local rollout<br>not enough| W
|
|
||||||
IA -->|Yes| AU{Is automatic\nupdate enabled?}
|
|
||||||
|
|
||||||
AU -->|Yes| CanUp{Can update?}
|
|
||||||
CanUp -->|No| ManIns
|
|
||||||
|
|
||||||
CanUp -->|Yes| Ins[Install]
|
|
||||||
Ins -->|Error| ManIns
|
|
||||||
Ins -->|Ok| R
|
|
||||||
|
|
||||||
AU -->|No| ManUp
|
|
||||||
ManUp -->|Ignore| W
|
|
||||||
|
|
||||||
|
|
||||||
F[Force update]
|
|
||||||
F --> NF
|
|
||||||
end
|
|
||||||
|
|
||||||
ManIns --> Web[Open web page]
|
|
||||||
NF --> Web
|
|
||||||
ManUp --> Web
|
|
||||||
R --> Re[Restart]
|
|
||||||
NF --> Q[Quit bridge]
|
|
||||||
NotifOrInstall -->|Automatic| W
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
The non-trivial is to combine the update with setting change:
|
|
||||||
* turn off/on automatic in-app updates
|
|
||||||
* change from stable to beta or back
|
|
||||||
|
|
||||||
_TODO fill flow chart details_
|
|
||||||
|
|
||||||
|
|
||||||
We are not support downgrade functionality. Only some circumstances can lead to
|
|
||||||
downgrading the app version.
|
|
||||||
|
|
||||||
_TODO fill flow chart details_
|
|
||||||
Reference in New Issue
Block a user