Solved problem with sieve ... Postfix + mysql+ postfixadmin + dovecot + sieve + amavisd (with spamassassin and clamav)

alfikmik

alfikmik

Hi there

got problem with sieve (it doesn't move mails to directories i mean spam to "Junk" folder.
In logs there is no information about any errors.
Spamassassin is called from amavisd and tags mail (as spam) properly but sieve doesn't move mails.

installed packages:
Code: 
amavisd-new-2.12.0_1,1         Mail scanner interface between mailer and content checkers
clamav-0.103.0,1               Command line virus scanner written entirely in C
dovecot-2.3.11.3_1             Secure, fast and powerful IMAP and POP3 server
dovecot-pigeonhole-0.5.11      Sieve plugin for the Dovecot 'deliver' LDA and LMTP
postfix-3.5.8,1                Secure alternative to widely-used Sendmail
spamassassin-3.4.4             Highly efficient mail filter for identifying spam

my configuration:

dovecot -n


Code: 
# 2.3.11.3 (502c39af9): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.11 (d71e0372)
# OS: FreeBSD 12.1-RELEASE-p2 amd64  nullfs
# Hostname: mydomain
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login
disable_plaintext_auth = no
first_valid_uid = 5000
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_debug = yes
mail_location = maildir:/var/vmail/%d/%n/
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric
    relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate
    mime foreverypart extracttext
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix =
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  mail_home = /var/vmail/%d/%n/
  sieve = file:/var/mail/%d/%n/sieve;active=/var/vmail/%d/%n/dovecot.sieve
  sieve_before = /usr/local/etc/dovecot/sieve/dovecot.sieve
  sieve_default = /usr/local/etc/dovecot/sieve/dovecot.sieve
  sieve_global = /usr/local/etc/dovecot/sieve/global/
  sieve_trace_debug = yes
  sieve_trace_dir = /var/log/sieve/
  sieve_trace_level = commands
}
protocols = imap pop3 lmtp sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}
ssl = no
userdb {
  args = /usr/local/etc/dovecot/dovecot-sql.conf
  driver = sql
}
protocol sieve {
  info_log_path = /var/log/dovecot-sieve.log
  log_path = /var/log/dovecot-sieve-errors.log
  managesieve_implementation_string = dovecot
  managesieve_max_line_length = 64 k
}
protocol lmtp {
  mail_plugins = " sieve"
}
protocol lda {
  mail_plugins = " sieve"
}

postconf -n


Code: 
alias_maps =
append_dot_mydomain = no
biff = no
broken_sasl_auth_clients = yes
compatibility_level = 2
content_filter = smtp-amavis:[127.0.0.1]:10024
home_mailbox = Maildir/
inet_interfaces = all
inet_protocols = ipv4
mailbox_command = /usr/local/libexec/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"
mailbox_size_limit = 0
message_size_limit = 102400000
mydestination = $myhostname localhost
myhostname = mx.mydomain.pl
mynetworks = 10.20.30.0/24 127.0.0.0/8
myorigin = xxxxx.com.pl
queue_directory = /var/spool/postfix
relay_domains = proxy:mysql:/usr/local/etc/postfix/mysql_relay_domains.cf
smtp_generic_maps = hash:/usr/local/etc/postfix/generic.hash
smtpd_banner = xxxxx.com.pl
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated,
    check_helo_access hash:/usr/local/etc/postfix/helo.whitelist, reject_unknown_client,
    reject_non_fqdn_hostname, reject_invalid_hostname, reject_unknown_hostname, permit
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated,
    reject_unauth_destination, reject_unknown_sender_domain, reject_unknown_recipient_domain,
    reject_non_fqdn_recipient, reject_non_fqdn_sender
smtpd_sasl_auth_enable = yes
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
transport_maps = proxy:mysql:/usr/local/etc/postfix/mysql_transport_maps.cf
virtual_alias_maps = proxy:mysql:/usr/local/etc/postfix/mysql_virtual_alias_maps.cf,
    proxy:mysql:/usr/local/etc/postfix/mysql_virtual_alias_domain_maps.cf,
    proxy:mysql:/usr/local/etc/postfix/mysql_virtual_alias_domain_catchall_maps.cf
virtual_gid_maps = static:5000
virtual_mailbox_base = /var/vmail/
virtual_mailbox_domains = proxy:mysql:/usr/local/etc/postfix/mysql_virtual_domains_maps.cf
virtual_mailbox_limit = 102400000
virtual_mailbox_maps = proxy:mysql:/usr/local/etc/postfix/mysql_virtual_mailbox_maps.cf,
    proxy:mysql:/usr/local/etc/postfix/mysql_virtual_alias_domain_mailbox_maps.cf
virtual_minimum_uid = 100
virtual_transport = dovecot
virtual_uid_maps = static:5000

postconf -M


Code: 
smtp       inet  n       -       n       -       -       smtpd
submission inet  n       -       n       -       -       smtpd
pickup     unix  n       -       n       60      1       pickup
cleanup    unix  n       -       n       -       0       cleanup
qmgr       unix  n       -       n       300     1       qmgr
tlsmgr     unix  -       -       n       1000?   1       tlsmgr
rewrite    unix  -       -       n       -       -       trivial-rewrite
bounce     unix  -       -       n       -       0       bounce
defer      unix  -       -       n       -       0       bounce
trace      unix  -       -       n       -       0       bounce
verify     unix  -       -       n       -       1       verify
flush      unix  n       -       n       1000?   0       flush
proxymap   unix  -       -       n       -       -       proxymap
proxywrite unix  -       -       n       -       1       proxymap
smtp       unix  -       -       n       -       -       smtp
relay      unix  -       -       n       -       -       smtp -o syslog_name=postfix/$service_name
showq      unix  n       -       n       -       -       showq
error      unix  -       -       n       -       -       error
retry      unix  -       -       n       -       -       error
discard    unix  -       -       n       -       -       discard
local      unix  -       n       n       -       -       local
virtual    unix  -       n       n       -       -       virtual
lmtp       unix  -       -       n       -       -       lmtp
anvil      unix  -       -       n       -       1       anvil
scache     unix  -       -       n       -       1       scache
postlog    unix-dgram n  -       n       -       1       postlogd
dovecot    unix  -       n       n       -       -       pipe flags=DRhu user=vmail:vmail
    argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${recipient}
smtp-amavis unix -       -       -       -       2       smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes -o max_use=20
127.0.0.1:10025 inet n   -       -       -       -       smtpd
    -o content_filter= -o local_recipient_maps= -o relay_recipient_maps=
    -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
    -o virtual_transport=dovecot

cat sieve/dovecot.sieve

Code: 
require ["envelope", "fileinto", "mailbox", "subaddress"];
if header :contains "X-Spam-Flag" "YES" {
        fileinto "INBOX.Junk";
        stop;
}
 
Yes, postfix, amavis, dovecot write information to log files / syslog, but there is no errors, or any information about passing emails to sieve.

IMHO It look likes after email pass thru amavis (amavis runs spamassassin and clam) sieve isn't fired.
 
Is your spamassassin is set up to put the header X-Spam-Flag ?
Check your /usr/local/etc/mail/spamassassin/local.cf

Or open the header of some e-mail and see if there's X-Spam-Flag
 
VladiBG said:
Is your spamassassin is set up to put the header X-Spam-Flag ?
Check your /usr/local/etc/mail/spamassassin/local.cf

Or open the header of some e-mail and see if there's X-Spam-Flag
Click to expand...
As I wrote before, yes email are tagged correctly...



Code: 
X-Spam-Flag: YES
X-Spam-Score: 5.135
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.135 tagged_above=2 required=4
    tests=[HTML_IMAGE_ONLY_24=1.282, HTML_IMAGE_RATIO_04=0.001,
    HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_BL=0.001, RCVD_IN_MSPIKE_L5=0.001,
    RCVD_IN_S5HBL=1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001,
    URIBL_ABUSE_SURBL=1.948, URIBL_BLOCKED=0.001]
    autolearn=no autolearn_force=no
 
INBOX.junk is subfolder of Inbox try to test the rule to /junk folder.

Code: 
if header :contains "X-Spam-Flag" "YES" {
        fileinto :create "Spam";
}

Or

Code: 
if header :contains "X-Spam-Flag" "YES" {

{
    fileinto "Junk";
    stop;
}

sieve-test(1)
 
I'll try this before but with no success BTW fileinto :create "Spam"; specially :create causes error and it seems is deprecated in never versions pigeonhole

That's why I have added:

Code: 
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes

to dovecot config
 
Last edited by a moderator:
Thanks for suggestion

sieve-filter -C -u <username@my.domain> /usr/local/etc/dovecot/sieve/dovecot.sieve 'INBOX'

works fine, also

sieve-filter -e -W -C -u <username@my.domain> /usr/local/etc/dovecot/sieve/dovecot.sieve 'INBOX'

do the job... but i want to do that immediately after email is received.

Any suggestions?
 
You have to reference Sieve/Pigeonhole from dovecot.conf (real excerpt from my Dovecot server):

Code: 
plugin {
        sieve = ~/.dovecot.sieve
        sieve_global_path = /usr/local/etc/dovecot/dovecot.sieve
}

The first sieve = line is local (per user) and the second sieve_global_path = is global (all users). You want to set the second, do something like:


Code: 
plugin {
        sieve_global_path = /usr/local/etc/dovecot/sieve/dovecot.sieve
}
 
Last edited by a moderator:
neel said:
You have to reference Sieve/Pigeonhole from dovecot.conf (real excerpt from my Dovecot server):

Code: 
plugin {
        sieve = ~/.dovecot.sieve
        sieve_global_path = /usr/local/etc/dovecot/dovecot.sieve
}

The first sieve = line is local (per user) and the second sieve_global_path = is global (all users). You want to set the second, do something like:


Code: 
plugin {
        sieve_global_path = /usr/local/etc/dovecot/sieve/dovecot.sieve
}
Click to expand...

which versions of sieve and dovecot You're using?

According to documentation (https://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Deprecated_Settings):

Deprecated Settings
These settings are deprecated in newer versions, but still recognized:

sieve_global_path = (< v0.2)
The deprecated name for the sieve_default setting.

sieve_dir = ~/sieve (< v0.3.1)
Directory for personal include scripts for the include extension. The Sieve interpreter only recognizes files that end with a .sieve extension, so the include extension expects a file called name.sieve to exist in the sieve_dir directory for a script called name. When using ManageSieve, this is also the directory where scripts are uploaded. For recent Pigeonhole versions, this location is configured as part of the sieve setting.

sieve_global_dir = (< v0.3.1)
Directory for :global include scripts for the include extension. The Sieve interpreter only recognizes files that end with a .sieve extension, so the include extension expects a file called name.sieve to exist in the sieve_global_dir directory for a script called name. For recent Pigeonhole versions, a more generic version of this setting is called sieve_global and allows locations other than file system directories.
Click to expand...

I have sth similar in my dovecot (all configs in first post in this thread):

Code: 
plugin {
  mail_home = /var/vmail/%d/%n/
  sieve = file:/var/mail/%d/%n/sieve;active=/var/vmail/%d/%n/dovecot.sieve
  sieve_before = /usr/local/etc/dovecot/sieve/dovecot.sieve
  sieve_default = /usr/local/etc/dovecot/sieve/dovecot.sieve
  sieve_global = /usr/local/etc/dovecot/sieve/global/
  sieve_trace_debug = yes
  sieve_trace_dir = /var/log/sieve/
  sieve_trace_level = commands
}
protocols = imap pop3 lmtp sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}
protocol sieve {
  info_log_path = /var/log/dovecot-sieve.log
  log_path = /var/log/dovecot-sieve-errors.log
  managesieve_implementation_string = dovecot
  managesieve_max_line_length = 64 k
}
protocol lmtp {
  mail_plugins = " sieve"
}
protocol lda {
  mail_plugins = " sieve"
}
 
I suppose sieve isn't triggered log dir for sieve is empty as I understand flow should be sth something like this:

Code: 
mail -> postfix       postfix -> dovecot-lda -> sieve
          |              ^
        amavis        amavis
          |              ^
        spamassassin & clamav
 
Last edited by a moderator:
What does the Dovecot startup line in dovecot.log look like? On my (simpler) setup, I get e.g.

Code: 
Dec 02 17:23:41 master: Info: Dovecot v2.3.11.3 (502c39af9) starting up for imap, lmtp, submission, sieve
 
Definitely dovecot-lda is not used. When I run ./dovecot-lda -d <email@mydomain> (in /usr/local/libexec/dovecot dir) there are entries in the dovecot logs for LDA, but when the mail goes normally postfix -> amavis -> postfix the LDA logs are silent.
 
Last edited by a moderator:
ok, problem solved :) ... as i supppose sieve wasn't triggered, specifically dovecot-lda ... this is the solution:

first of all i've checked smtp -v (postfix master.cf)

Code: 
127.0.0.1:10025 inet    n       -       -       -       -       smtpd -v
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8,10.20.30.0/24
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

but nothing interested here was found

as i have virtual_mailbox stored in mysql i start debugging queries for mysql:

mysql -u <user> -p
Code: 
SET global log_output = 'FILE';
SET global general_log_file='/tmp/mysql.log';
SET global general_log = 1;
exit

after than:
tail -F /tmp/mysql.log

that i catch all queries during mail processing, some interesting query was:

Code: 
SELECT transport FROM domain WHERE domain='my.domain' AND active = '1';

the returned transport value was virtual so i update it to dovecot
mysql -u <user> -p
Code: 
USE postfix
UPDATE domain SET transport='dovecot' WHERE domain='my.domain';


now everything works like a charm
 
You must log in or register to reply here.
Back
Top