Bug 2627 - Filtering does not work on NNTP
Summary: Filtering does not work on NNTP
Status: RESOLVED DUPLICATE of bug 1920
Alias: None
Product: Claws Mail (GTK 2)
Classification: Unclassified
Component: Filtering (show other bugs)
Version: 3.8.0
Hardware: PC Linux
: P3 normal
Assignee: users
URL:
Depends on:
Blocks:
 
Reported: 2012-03-13 16:13 UTC by flobber
Modified: 2014-04-27 12:56 UTC (History)
0 users

See Also:


Attachments

Description flobber 2012-03-13 16:13:31 UTC
The global filters (Configuration -> Filtering) are not applied to new news articles (NNTP).

I tried to filter on something like "if from is in addressbook trolls then mark as read" to implement something like a kill-file.  Unfortunately, it only works on mail accounts but not on NNTP accounts.

Setting that rule as a folders's processing rules does work, but then Claws scans ALL messages of that folder (read and unread) each time I open it.  That takes too long on newsgroups with thousands of messages.  Processing on start would not help as I usually never quit Claws-Mail.

Is it possible to filter NNTP articles or alternatively process only NEW articles?
Comment 1 Manfred Schmitt 2012-03-14 14:44:44 UTC
It should work if you use Pre-processing instead of filtering so I would not consider this as a bug. At least Pre-processing works here to color messages from me and set the "watch" mark on the thread (not only) in news.
Comment 2 flobber 2012-03-15 20:41:06 UTC
Yes, it DOES work with processing rules (and also as a pre-processing rule) but (pre-)processing always processes ALL messages (old and new ones) and that takes very long when there are more than a couple of messages.  I.e. opening a newsgroup with 8700 messages takes about 4 seconds hot-cache and 12 seconds cold-cache with the single rule (6 entries in the addrbook):

  found_in_addressbook "From" in "addrbook-killfile.xml" ignore

Btw, using pre-processing rules is even worse than folder-specific processing rules as they are global and are applied to all folders (even trash), so entering any folder with a significant number of messages is slow.

I only want to filter/process NEW messages (usually only a handful) so that there's no noticable delay when opening them.
Comment 3 flobber 2012-03-15 21:02:27 UTC
Ok, I think I found a solution.  The slow part seems to be the addressbook match.  If I guard the rule with a "new" (new & found_in_addrbook ...) the processing is much faster (but it still scans all messages).  At least the hot-cache case is acceptable now - I couldn't check the cold-cache case yet.

Regarding whether the not-working "Filter" is a bug or not: The NNTP-specific "Receive Tab" in the account preferences has a "Filter messages on receiving" checkbox.  It is checked but the filter is not working - looks like a bug ;-)
Comment 4 Manfred Schmitt 2012-03-15 23:06:37 UTC
Hm, I don't see a real difference wheter my few kill/watch rules are enabled or not, entering, for example, de.comp.os.unix.linux.misc with > 10000 messages takes roughly 1-2 seconds. 
Yes, that's cached, it will be bit slower when there are new messages. But it's not that slow here, you haven't enabled the filtering log, don't you?
Especially the filtering log is a really really huge performance killer.

Bug or not: Indeed, regarding the configuration filtering should also work on NNTP, point taken.
Comment 5 flobber 2012-03-16 00:42:21 UTC
> you haven't enabled the filtering log, don't you?

Filtering log is disabled.

> 10000 messages takes roughly 1-2 seconds.

Yes, it's really the "found_in_addressbook" pattern that's slow.  If I limit it to "new" messages:

  new & found_in_addressbook "From" in "addrbook-killfile.xml" ignore

entering folders becomes as nearly as fast as if (pre-)processing is disabled.


But I noticed one thing that will still make a working "Filtering" my preference: Processing rules are only executed when opening a folder.  So the folder list may show new messages when in fact the processing rules will mark all of them as "ignore" when I open the folder.  Filtering messages on receive would not falsely hilight the folder list, trigger tray-icons, etc.
Comment 6 Andrej Kacian 2012-03-27 00:50:33 UTC
Hi, I too agree that NNTP accounts should be subject to filtering. Meanwhile, I'm using a somewhat dirty workaround - in folderlist.xml file, I change folder type from "normal" to "inbox" for each newsgroup folder I want filtering to be active in.

Then, to implement a kill list, I have one rule which looks like:
from matchcase "abc@def.ghi" | from matchcase "zxc@asd.qwe" | from matchcase ...

Rule action is "mark_as_read hide". Do not delete the unwanted messages, or they will be redownloaded later. :)
Comment 7 flobber 2012-03-28 01:03:43 UTC
Nice trick with the "inbox" type.  So it seems, Claws-Mail actually *can* filter newsgroups only the check is wrong (only folders of type "inbox" whereas it should be something like: all folders where filtering is enabled).

Anyway, in the meantime I've already disabled (pre-)processing for NNTP as that triggers downloading of all articles.  I just tried the "inbox" trick and it seems that filtering does the same (downloads all articles - correct me if I'm wrong), something I assumed would not be necessary for filtering (at least as long as there's no rule which needs the body).

Does somebody know, whether there's a hook that I could use to write my own filter plugin?  A hook, that is called for each new article/mail _header_ which does not trigger downloading the bodies?
Comment 8 boxcars 2014-01-26 19:51:53 UTC
This bug is a dupe of bug 1920.
Comment 9 Paul 2014-04-27 12:56:55 UTC

*** This bug has been marked as a duplicate of bug 1920 ***
Comment 10 users 2018-12-30 00:36:19 UTC
Changes related to this bug have been committed.
Please check latest Git and update the bug accordingly.
You can also get the patch from:
http://git.claws-mail.org/

++ ChangeLog	2018-12-30 00:36:17.442809084 +0100
http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=6408f4a501cf08f6e33bb920b8bb4e51dddb23db
Author: Stephan Sachse <ste.sachse@gmail.com>
Date:   Sun Jul 26 02:22:15 2015 +0200

    fix nntp manually filtering with account specific rules
    
    manually running filter on nntp folders doesn't work correctly with
    account specific rules. filtering_match_condition() in filtering.c:568
    always gets a NULL as ac_prefs and every check with ac_prefs fails
    subsequently. later cur_account is used instead, what is imo clearly
    wrong and useless.
    
    i have patched summaryview.c:5649 summary_filter() to extract the
    account data from summaryview and hand over them as new parameter to
    summary_filter_func().
    
    works for me on fedora 22 and claws-mail-3.11.1
    
    Signed-off-by: Stephan Sachse <ste.sachse@gmail.com>
    Signed-off-by: Andrej Kacian <ticho@claws-mail.org>

http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=2a3e98d24814b8a92b0d995c94a4c24c580a4c4a
Author: Stephan Sachse <ste.sachse@gmail.com>
Date:   Sun Jul 26 01:34:55 2015 +0200

    fix nntp filtering on incoming messages
    
    this should fix bug #1920 and bug #2627
    
    folder_item_scan_full() in folder.c at line 2343 applies filter only to
    folder items of special type F_INBOX. but nntp folder items are of
    special folder item type F_NORMAL and folder type F_NEWS. so i change
    the condition for do_filter to also match nntp folders. works for me on
    fedora 22 and claws-mail-3.11.1
    
    Signed-off-by: Stephan Sachse <ste.sachse@gmail.com>
    Signed-off-by: Andrej Kacian <ticho@claws-mail.org>

Note You need to log in before you can comment on or make changes to this bug.