Sending documents by e-mail
Foresi sends an invoice as a link to its PDF, through the firm’s own mail server. The customer gets a short mail with the link; the link opens a page with the document, its payment details and the PDF. This page describes how to set sending up, what it keeps, and what its states mean.
What happens when you send
Section titled “What happens when you send”-
The send dialog opens over the page it starts from: the invoice list, the history of an invoice, or the invoice form after “Send and save”. It shows the recipients and the message at a glance, with Send ready for one click or Enter; “Edit text” opens the subject, the text and other recipients in the same dialog. Everything is filled in from the firm’s template in the document’s language. Without JavaScript the same address is a page of its own.
{Link to the document}marks where the link goes; it is the only placeholder left, and any other brace in the text is sent as written. -
On Send, Foresi draws the PDF and keeps it, together with the subject, the text, the recipients and the payment details, exactly as they will be sent. A later edit of the invoice changes none of this.
-
The send waits in the
mailqueue. A worker hands it to the firm’s mail server. A temporary refusal is retried after 1, 4, 16, 64 and 256 minutes (about 5.7 hours, which outlasts greylisting); a permanent refusal (a 5xx answer, a refused login) fails at once.The server takes or refuses a send as a whole. If it refuses one recipient (a 5xx answer to
RCPT TO, for example an unknown Cc address), the whole send fails and no recipient gets it, the customer included. The reason names the refused address: correct or remove it and send again. -
The invoice list marks each invoice as sending, sent, read or failed. The invoice’s history shows every send with its text and the PDF as it was sent, and Sales › Sent e-mails lists all sends with a filter by firm and state.
Every send has a purpose: an invoice, or a reminder with its step number. Sales › Sent e-mails filters by it, and the invoice’s history names the reminder step. Sends made by the daily runs, recurring invoices and automatic reminders, are the system’s: they skip the per-user minute limit and take at most 80% of the firm’s daily budget, so people keep room to send by hand. See recurring.md and reminders.md.
A reminder can lose its reason while it waits in the queue. Just before it leaves, Foresi locks the invoice and cancels the reminder when the invoice was paid or moved to the trash meanwhile; an automatic one is also cancelled when the firm switched reminders off or the contact is no longer reminded. A cancelled reminder shows the state Cancelled. An invoice send is never cancelled this way.
“Copy to me” sends a second message, with a link of its own, to the firm’s contact e-mail (or the signed-in user’s), so opening your own copy never marks the customer’s link as read.
Setting up the mail server
Section titled “Setting up the mail server”Each firm sends through its own SMTP server. The owner sets it under My firms › (firm) › E-mails › Mail server: host, port, encryption (STARTTLS on port 587, or TLS on 465), user name, password, sender address and name. “Send a test e-mail” sends one message to the signed-in owner. There is no shared fallback server: a firm without a server cannot send, and the dialog says where to set it.
- The connection is always encrypted and the certificate is always checked. There is no unencrypted mode.
- A host that resolves to a private, loopback or link-local address is refused, when saving and again before connecting, unless the operator sets
app.mail.allow_private_hosts: true(for a mail relay inside the operator’s own network). Fromis the server’s sender address.Reply-Tois the firm’s contact e-mail when it has one, so replies reach the firm.
Passwords: MAIL_SECRET_KEYS
Section titled “Passwords: MAIL_SECRET_KEYS”Mail server passwords are sealed with sodium_crypto_secretbox before they are stored. The keys come from the environment variable MAIL_SECRET_KEYS, never from the database and never from APP_SECRET, so a copy of the database alone does not reveal them.
Generate a key (32 random bytes, base64):
php -r 'echo base64_encode(random_bytes(32)), "\n";'Set it outside committed files, for example in .env.local or the service environment:
MAIL_SECRET_KEYS=k1:BASE64KEYThe value is a comma-separated list of id:key. The first key seals new passwords; the others only open what they sealed. To rotate, put a new key first (k2:NEWKEY,k1:OLDKEY), ask each firm owner to save the mail server password again, and remove the old key once no stored password uses it. Without any key no firm can save a mail server. A password that no configured key opens fails the send with “The mail server password cannot be opened with the configured keys”; the owner enters it again.
SPF, DKIM and DMARC
Section titled “SPF, DKIM and DMARC”Foresi does not sign mail and does not choose the route a mail takes: the firm’s SMTP provider does. Whether the customer’s mail server accepts the mail depends on the DNS records of the sender’s domain. For a sender address invoices@example.com sent through a provider:
- SPF: the domain’s SPF record includes the provider’s servers, as the provider documents it, for example
example.com. TXT "v=spf1 include:_spf.mailprovider.example ~all". - DKIM: the provider signs the mail. Add the DKIM record the provider gives you, for example
selector1._domainkey.example.com. TXT "v=DKIM1; k=rsa; p=MIIB...". Foresi has no DKIM key of its own and needs none. - DMARC: the domain of the
Fromaddress must align with the SPF or DKIM domain. Start with monitoring, for example_dmarc.example.com. TXT "v=DMARC1; p=none; rua=mailto:dmarc@example.com", and tighten the policy once the reports are clean.
The sender address in the mail server settings must be one the provider allows for that account.
The link base: DEFAULT_URI
Section titled “The link base: DEFAULT_URI”Links in sent mail are built from DEFAULT_URI, the public address of the installation, for example DEFAULT_URI=https://invoices.example.com. Sending is refused when it points to localhost or a loopback address (the customer could not open it) and, in production, when it is plain http.
The running worker
Section titled “The running worker”Sending needs the background worker, which must consume the scheduler and mail transports beside the others:
php bin/console messenger:consume scheduler_foresi mail events async webhooks registry --memory-limit=256M --time-limit=3600scheduler_foresiticks every five minutes and runs the timed jobs of each organisation, such as handing back sends whose queue message was lost. Without it, a send whose web request died after storing it is never retried.maildelivers the sends. Without it nothing leaves; after 15 minutes the application shows a notice that sends are waiting.- A worker claims a send for five minutes while it delivers it, so no second worker mails the same send meanwhile. A conversation with the mail server is cut off after 90 seconds in all, and each wait for the server after 30, so the customer’s mail and the firm’s copy both end well inside the claim, however slowly the server answers. A cut-off conversation counts as a temporary failure and is retried.
- The schedule’s state is kept in the application cache and its lock in
LOCK_DSN. The default file lock (flock) suits one host; with several hosts set a shared lock store, as for imports (see import.md).
See import.md for systemd and supervisor units.
The public link
Section titled “The public link”The link is https://<DEFAULT_URI>/d/<token>. The token has 256 bits; the database keeps only its SHA-256 hash, so a copy of the database cannot open links. The page and the PDF need no sign-in, start no session, run no script and set X-Robots-Tag: noindex, nofollow, Referrer-Policy: no-referrer and Cache-Control: private, no-store. An unknown, revoked or cancelled link gets the same answer.
The public link is limited to 60 requests a minute per client IP address, counted together for the page, the PDF, “Open PDF”, “Download” and any other address under /d/. The client IP is the one Symfony gives (Request::getClientIp()), so behind a reverse proxy or a load balancer trusted_proxies must be set: SYMFONY_TRUSTED_PROXIES (or framework.trusted_proxies) to the proxy’s addresses, and framework.trusted_headers to the headers it sets, such as x-forwarded-for, if they are not Symfony’s defaults. Without it every request comes from the proxy’s address, all customers share one limit, and one busy minute answers them all with 429.
Once the invoice is paid or overpaid, the page says “Paid” and shows no account, no QR code and no other way to pay, so the customer cannot pay twice; the PDF keeps the payment details it was sent with, and the page notes the date it was paid and that nothing is left to pay. After a part payment the page shows what is left to pay, without the QR code, which would ask for the whole amount. See bank-pairing.md.
A link keeps showing the PDF that was sent, also after the invoice is edited; the invoice form warns that the invoice changed since it was sent, so you can send it again. Trash cancels sends that have not gone out and revokes their links. A delivered invoice cannot be trashed, so to end a link that went to a wrong address, the operator runs:
php bin/console app:invoice:revoke-link <send-id>The send id is in the address of the “PDF as sent” link on the invoice’s history.
What the states mean
Section titled “What the states mean”| State | Meaning | What it does not mean |
|---|---|---|
| Sending | Stored and waiting for the worker, or being retried after a temporary refusal | That the worker runs |
| Sent | The firm’s mail server accepted the message. The invoice is then marked delivered | That it reached an inbox. A later bounce goes to the firm’s own mailbox, as the envelope sender is the firm’s address; Foresi does not read it |
| Read | Someone opened or downloaded the PDF from the customer’s link, by a click | That the customer read it. Opening the page, a mail filter following the link, the firm’s own copy and a signed-in member of the firm never count |
| Failed | The server refused the message for good, or the retries ran out. The reason is shown in the user’s language, with the SMTP code | Anything about the recipient beyond that answer |
| Cancelled | The send did not leave: the invoice went to the trash before it did, or, for a reminder, it was paid meanwhile or is no longer to be reminded | That anything reached the customer |
There is no tracking pixel and no IP address is stored.
Limits
Section titled “Limits”| Limit | Value |
|---|---|
| Sends per signed-in user | 30 a minute |
| Sends per firm | 500 a day, of which the system’s daily runs take at most 400 (80%) |
| Test e-mails per user | 5 a minute |
| Public link (page and PDF) per client IP address, see the public link | 60 a minute |
| Recipients per send (To and Cc) | 15 |
| Other recipients typed in the dialog | 5 |
| Subject | 255 characters |
| Text | 10,000 characters |
Backups
Section titled “Backups”The PDFs as sent are kept in <APP_SHARE_DIR>/<environment>/sent-documents (var/share/prod/sent-documents by default), each under its organisation and with its SHA-256 on the send. Back this directory up together with the database: a send without its file cannot show its PDF, and a file whose hash no longer matches is refused.
Personal data
Section titled “Personal data”This section is subject to professional legal review.
The send log holds the recipients’ addresses, the subject, the text, the PDF as sent and the moment the link was first opened. This is personal data. It is kept with the invoice for the invoice’s retention period, as described in ADR 0011, because it is the record of what the customer received. Foresi stores no IP address, uses no tracking pixel and builds no profile of a person. After the retention period the addresses and the text may be pseudonymised under the same ADR. During the period the log is not erased on request; the operator states this in their privacy notice, including that opening a link is recorded.
Accepted differences from the plan
Section titled “Accepted differences from the plan”These were decided while sending was built and replace the plan where they differ:
- Read counts only an open or a download the user started (a click, or a request the browser marks as user-initiated). The PDF embedded in the page never counts. An honest read state was preferred to a higher count.
- The link page shows the payment details and the QR code of the document, and drops the repeated document number, since customers pay from it.
- Amounts in Czech crowns read
Kčon a Czech document (PDF, link page, mail) andCZKon an English one. ISDOC keeps the ISO code. - The send dialog’s text is filled in before it is shown, so only the link is a placeholder there; braces in a customer’s name or a description stay text.
- A refused send puts the keyboard focus on the first field to fix, as Foresi’s other forms do; the error summary takes the focus only for a refusal that names no field.
- The form warns about a change after sending from the invoice’s history (a newer version and a change from the second of the send on), so the delivery a send records does not count as a change.
- The reason of a failed send is stored as a code with the SMTP code and shown in the user’s language. Sends from before keep their English text.
- iDoklad’s own send dialog was not opened while this was built, to avoid mailing a real customer; the dialog follows iDoklad’s help.