forked from Silverfish/proton-bridge
143 lines
4.5 KiB
Markdown
143 lines
4.5 KiB
Markdown
# Integration tests
|
||
|
||
This folder contains integration tests of the Bridge app.
|
||
|
||
## What and how we are testing
|
||
|
||
```mermaid
|
||
graph LR
|
||
S[Server]
|
||
C[Client]
|
||
U[User]
|
||
Creds[Credentials store]
|
||
|
||
subgraph "Bridge app"
|
||
Core[Bridge core]
|
||
Store
|
||
Frontend["Qt / CLI"]
|
||
IMAP
|
||
SMTP
|
||
API[PMAPI]
|
||
|
||
IMAP --> Core
|
||
SMTP --> Core
|
||
Frontend --> Core
|
||
Store --> Core
|
||
Core --> API
|
||
end
|
||
|
||
C --> IMAP
|
||
C --> SMTP
|
||
U --> Frontend
|
||
API --> S
|
||
Core --> Creds
|
||
```
|
||
|
||
We want to test Bridge app from outside as much as possible. So we mock server (API),
|
||
credentials store and call commands to IMAP or SMTP the same way as client would do.
|
||
|
||
## Running tests
|
||
|
||
In order to run Integration tests just go into the test folder `cd test`
|
||
and run `make test`.
|
||
|
||
You can also test only specific feature (or subset of features) by using `FEATURES` environment
|
||
variable: `FEATURES=features/imap/message/create.feature make test`.
|
||
|
||
## Example test
|
||
|
||
BDD test in gherkin (cucumber) format (https://cucumber.io/docs/gherkin/reference/).
|
||
|
||
```
|
||
Feature: IMAP update messages
|
||
Background:
|
||
Given there is connected user "user"
|
||
And there are messages in mailbox "INBOX" for "user"
|
||
| from | to | subject | body | read | starred |
|
||
| john.doe@mail.com | user@pm.me | foo | hello | false | false |
|
||
| jane.doe@mail.com | name@pm.me | bar | world | true | true |
|
||
And there is IMAP client logged in as "user"
|
||
And there is IMAP client selected in "INBOX"
|
||
|
||
Scenario: Mark message as read
|
||
When IMAP client marks message "1" as read
|
||
Then IMAP response is "OK"
|
||
And message "1" in "INBOX" for "user" is marked as read
|
||
And message "1" in "INBOX" for "user" is marked as unstarred
|
||
```
|
||
|
||
Is translated into code with godog (https://github.com/cucumber/godog/).
|
||
|
||
```go
|
||
// Registration
|
||
func FeatureContext(s *godog.Suite) {
|
||
s.Step(`^there is connected user "([^"]*)"$`, thereIsConnectedUser)
|
||
}
|
||
|
||
// Godog step function
|
||
func thereIsConnectedUser(username string) error {
|
||
account := ctx.GetTestAccount(username)
|
||
if account == nil {
|
||
return godog.ErrPending
|
||
}
|
||
ctx.GetPMAPIController().AddUser(account.User, account.Addresses)
|
||
return ctx.LoginUser(account.Username(), account.Password(), account.MailboxPassword())
|
||
}
|
||
```
|
||
|
||
## BDD
|
||
|
||
BDD has three parts:
|
||
|
||
* `Given` (setup),
|
||
* `When` (action)
|
||
* and `Then` (check).
|
||
|
||
Setup has to prepare context and always end without error. Action, on
|
||
the other hand, needs to always end without error, but store it in
|
||
the context. Check should analyze the status of the bridge, store or
|
||
API and also check whether something failed before.
|
||
|
||
Therefore we cannot use a sentence such as `there is user` for both
|
||
setup and check steps. We always begin setup steps with `there is/are`,
|
||
while check steps are written in the form `something is/has feature`.
|
||
Actions are written in the form `something does action`. By doing this
|
||
we can always be sure what each steps does or should do.
|
||
|
||
In the code, we separate those parts in its own files to make sure
|
||
it's clear how the function should be implemented.
|
||
|
||
In the `Given` phase is also generally better to setup data (as `there are messages...`)
|
||
first, then users (`there is connected user...`) and then connections (`there is IMAP client...`).
|
||
This can prevent some hitches in internal implementation of integration tests.
|
||
|
||
## API faked by fakeapi or liveapi
|
||
|
||
We need to control what server returns. Instead of using raw JSONs,
|
||
we fake the whole pmapi for local testing. Fake pmapi behaves as much
|
||
as possible the same way as real server, but does not follow every
|
||
single detail. Otherwise we would end up with writing complete server. :-)
|
||
|
||
For both -- fake local pmapi and real live server -- we use controller.
|
||
Controller is available on test context and does setup like setting up
|
||
internet connection, user settings, labels or messages.
|
||
|
||
Accounts for each environment are set up in `accounts` folder. Each
|
||
test function should use `TestAccount` object obtained by test ID
|
||
(such as `user` or `userMultipleAddress` for users, or `primary`
|
||
or `secondary` for addresses) and use available functions to get real
|
||
IDs (even if fake API uses the test IDs as real ones).
|
||
|
||
Testing against live is using real users and doesn't work in parallel.
|
||
Only one job against live at a time can be running.
|
||
|
||
## External e-mail accounts
|
||
|
||
We have some external accounts which we are using for testing:
|
||
|
||
* pm.bridge.qa@gmail.com
|
||
* bridge-qa@yandex.ru
|
||
* bridgeqa@seznam.cz
|
||
|
||
For access, ask bridge team.
|