TC forum mails are taking to long

English support forum

Moderators: white, Hacker, petermad, Stefan2

User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: TC forum mails are taking to long

Post by *Usher »

2ghisler(Author)
My provider restored hidden fields in server config. Now I can see that SpamAssassin is misconfigured, but it's still good enough. Current score is -98.4 = -100 (whitelist) + 1.6 (SpamAssaSsin rules).

General rules involved:
* LOCAL_SUBJECT_UTF8 - As I have already mentioned, US-ASCII characters in any case should not be encoded. It may happen to German, French, Dutch, Spanish, Czech or Slovak text (if accents, umlauts etc. are present), but English text is fully 7-bit.

* TXREP - It's sender reputation, replaces auto whitelist. If I understand correctly, it may be trained locally to recognize spam and ham or use some external IP/domain blocking list. The docs claim also that this rule prefers DKIM rather than SPF. See here:
https://wiki.apache.org/spamassassin/TxRep
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_TxRep.html

My providers uses also localized set of rules called ZABOJCASPAMU, so I can't do better check on this server.

Don't change anything for now, please. I have found another email provider, which most likely uses only general rules, so I will start new tests and report results later.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: TC forum mails are taking to long

Post by *ghisler(Author) »

LOCAL_SUBJECT_UTF8: The subject is sent in plain text if it contains only English and no question mark, and is shorter than 60 characters, with this code:

Code: Select all

if (strlen($string)<=60 && strpos($string,"?")==false && imap_8bit($string) == $string)
       return $string;
TXREP: We are not on any blacklists and use DKIM and SPF. Any way to improve that?
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: TC forum mails are taking to long

Post by *Usher »

2ghisler(Author)

1. English letters, numbers and basic punctuation marks (available on the keyboard) are printable 7-bit US-ASCII characters and in ANY case they should NOT be encoded, even in lines 998 byte long (max width in standard) with many lines of continuation (no limit in standard). NEVER EVER.

And there are no us-ascii characters disallowed in Subject, it's an unstructured field. See RFC 5322:
RFC 5322 p.2.2 wrote:A field body may be composed of printable US-ASCII characters
as well as the space (SP, ASCII value 32) and horizontal tab (HTAB,
ASCII value 9) characters (together known as the white space
characters, WSP). A field body MUST NOT include CR and LF except
when used in "folding" and "unfolding"
Some people prefer to configure forum to limit Subject line length in order to fit in more strict 78 byte line length limit (78-9=69 bytes of text) and avoid long line folding in emails.

2. I suggested to replace non-standard "X-mailer" field with its standard equivalent "User-Agent" rather than just delete the first one.

3. DKIM is currently not configured. I can see only non-standard field:

Code: Select all

X-CDC-DKIM: signer='ghisler.ch' status='none' reason=''
I don't know if you can find proper features in PHP to generate field like quoted below:

Code: Select all

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=disqus.net; s=mta;
	t=1546952458; bh=iG2z6eMqfcwZz71/GRZKuMRTnD5vx1gtVORGC60GxGk=;
	h=Subject:From:To:Date:Reply-To;
	b=uHA6i3KUGzN95Jy+OyBv2myreE4O+76WQlf+JfPfw6QnrQd9tSjdgoVHH8xdCB3m3
	 SfPLCJxKBmbgk4GZBpFo9vxcc9NtDsWPliUSOvWZuCRqds/0SqIirFbTuqhUJQ3KQs
	 N4QFNDs+OYywMGJK2bUwpqG/btCsLd7ZnSOaCFbtUnYgRqzsOezd/OswDOyZuqanjq
	 yFGXNEI/XhrXdufOliWpqmK21uANVLc+6f80Ast+fKNNXN/vIHG1gGLW/aCLKyibDG
	 9WfNqkpRLoVzE1PnOejMs1IAMUydk/3jj61pzIYBwxMYtvxrN0J34EJEvutYP8J5M9
	 Gu7Me04VYpXNA==
I don't remember all parameters, they are described in RFC 6376 standard, but it's obvious that some header fields are signed (see h= …).

4. I have read some topics about SpamAssassin (SA) on a few forums and it looks like most people don't understand what they do. They use SA because someone have recommended it, but they don't care to configure it properly, just add some filters and rules at their own discretion, without any tests on real spam/ham messages. So it won't be better, I'm afraid. Whatever rule with non-zero score will be involved, it will cause message delivery delay. Notice that negative score (f.e. whitelisting) may also cause delay.

So just fix the first three points and forget it.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: TC forum mails are taking to long

Post by *ghisler(Author) »

1. OK, I have changed the limit to 900 now, I hope it works.
2. I have now added
User-Agent: phpBB3
3. That's odd, it is enabled in Plesk as described here:
https://docs.plesk.com/en-US/onyx/customer-guide/mail-settings/enabling-dkim-email-signing.74718/
I do use the plesk DNS server. Any ideas?
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: TC forum mails are taking to long

Post by *Usher »

2ghisler(Author)
Ad. 1, 2. OK now. I suppose that you should also check subject length limit on forum in phpBB settings. I remember that some forum used very strict limit 128 chars only…

Ad 3. I know that DKIM is enabled. It's NOT configured. There is still the same header field:

Code: Select all

X-CDC-DKIM: signer='ghisler.ch' status='none' reason=''
…though it should look like that (even ransomware can use DKIM!):

Code: Select all

X-CDC-DKIM: signer='broadmanholmanpodcast.com' status='pass' reason=''
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=mail; d=broadmanholmanpodcast.com;
 h=From:Subject:To:Content-Type:MIME-Version:Reply-To:Date; i=admin@broadmanholmanpodcast.com;
 bh=rAzNQqQIy60kMDoNScZP/AVu3gsOuO9Z7/gENjbjUK4=;
 b=d9EFH9FHPaJZYTlWSyFZ1bX3UF4s2ExLDwmvrpKceYt9mVF/ynmTi1zRgnRDVduPTtgxTY9P7Rn7
   y0s6sv1+fOHYpmWM1cBStiN612ba2BdjlYS3L6Y+ezMBu2h8okc1CAIBdM0IN47iUuoGOO6Wd6+a
   30ud7Btx1mPkonxfTU8=
But I'm not a DKIM guru, I can only guess…

The standard allows to choose some header fields to be signed so you must at least go to the configuration page and accept the settings there, if such a page exists.
Or create proper file with DKIM policy (see the linked Plesk docs) and put it in the proper place.
Or create and add DKIM policy record to DNS zone of the DNS server (see the linked Plesk docs again). See search results: https://google.com/search?q=dkim+policy+record+syntax
Or put the generated keys in proper place (but they seem to be already there).
Or anything else…

Just one more thing. SPF seems to be turned off now, but you can enable both SPF and DKIM together.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: TC forum mails are taking to long

Post by *ghisler(Author) »

Author of Total Commander
https://www.ghisler.com
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6449
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Re: TC forum mails are taking to long

Post by *Horst.Epp »

As Thread starter I'm watching the discussion and changes but I don't see any better performance.
Still the same, a new entry in a Google group is in a few seconds max minutes in my web.de inbox
while all TC forum mails are already known to me because I have seen them in the forum long before the mail arrived.
I can live with that but something magic must be there.
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: TC forum mails are taking to long

Post by *Usher »

There are workarounds mentioned there in the docs. And it looks like I have already suggested something similar - use PHPmailer or find mods for phpBB… I'm pretty sure I have seen messages sent from PHPMailer with valid DKIM signature.
If you don't want to test workarounds now, you should turn off DKIM and go back to SPF.
Last edited by Usher on 2019-01-22, 18:09 UTC, edited 1 time in total.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: TC forum mails are taking to long

Post by *Usher »

Horst.Epp wrote: 2019-01-22, 16:23 UTCI can live with that but something magic must be there.
It's a magic called "PHP", I'm afraid.
There are may sites using outdated or unpatched PHP software and there are many unprotected (or poorly protected) PHP-based forums.
That way PHP is commonly abused by people sending spam or malware.
Then antispam and antimalware devs add rules to filter emails sent by PHP or containing links with PHP requests.
And this protection software may be also misconfigured…

So in such cases you can either live with annoyances or search for a better provider.

BTW. As you know, protection on this forum is also poor and the forum is not spam-free. All the messages from new users should be moderated (and it's not enough to moderate only first message).
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: TC forum mails are taking to long

Post by *ghisler(Author) »

protection on this forum is also poor
What do you mean?
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: TC forum mails are taking to long

Post by *Usher »

2ghisler(Author)
It should be "spam protection" there, and it should be clear when reading explanation which follows those words in my previous message, I suppose.
Just open Moderator Control Panel, go to "Moderator logs" and watch how many spam messages are deleted every day. I'm pretty sure that all of them are sent by new users. Well, almost all. If spam volume is large enough, it really may be better to moderate all new users.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: TC forum mails are taking to long

Post by *ghisler(Author) »

I want to avoid moderating messages on the board before they are posted. It would delay all answers considerably
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: TC forum mails are taking to long

Post by *Usher »

2ghisler(Author)
I'm talking only about users which post their first messages. If the forum hadn't implemented such a feature yet, it shouldn't be hard to do it, something like:

Code: Select all

if posts(user) <= 3
   moderate()
endif
It can't be simpler…
1 is too low in most cases, 10 may be too high - more moderators are needed.
Andrzej P. Wozniak
Polish subforum moderator
Post Reply