Bug 3212 - When msgnum matches a sub-folder name, fetch fails
Summary: When msgnum matches a sub-folder name, fetch fails
Status: RESOLVED FIXED
Alias: None
Product: Claws Mail (GTK 2)
Classification: Unclassified
Component: Folders/IMAP (show other bugs)
Version: 3.9.3
Hardware: All Linux
: P3 normal
Assignee: users
URL:
: 3290 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-06-15 04:14 UTC by Jeroen Roovers
Modified: 2014-09-25 22:07 UTC (History)
1 user (show)

See Also:


Attachments
claws-mail --debug 2>&1 | tee claws-mail.debug.log (240.92 KB, text/plain)
2014-06-16 16:47 UTC, Jeroen Roovers
no flags Details

Description Jeroen Roovers 2014-06-15 04:14:14 UTC
** (claws-mail:9482): WARNING **: [02:47:27] IMAP error on *****: FETCH error


(claws-mail:9482): Claws-Mail-WARNING **: can't fetch message 2011


(claws-mail:9482): Claws-Mail-WARNING **: can't get message file path.

This happened because the message in question resided in an IMAP folder that contained two sub-folders: 2011 and 2012[1].

And indeed, the other message I failed to retrieve had msgnum = 2012:

** (claws-mail:9482): WARNING **: [02:47:36] IMAP error on *****: FETCH error


(claws-mail:9482): Claws-Mail-WARNING **: can't fetch message 2012


(claws-mail:9482): Claws-Mail-WARNING **: can't get message file path.

After renaming the two offending IMAP folders, the imapcache was able to store two files named "2011" and "2012" along with the renamed folders.

I think this namespace collision should be fixed by changing the file naming scheme. Working around the problem by copying the message or renaming the folder isn't something I'd want to often do, and as you might understand I have many IMAP folders named after years.

[1]  Those are years - I store messages in those that I received in those years, respectively. I wouldn't think it's unlikely someone names IMAP folders after the year they received the messages they store in those folders, so I'm probably not alone.
Comment 1 Pierre Fortin 2014-06-15 04:52:29 UTC
Namespace collision looks like a good catch...

I too have lots of 20xx folders[1]...  sounds like flagging the directory names is needed, such as adding a:
- leading "." -- I don't like this one since that 'hides' those names.
- trailing ".d" (very common in Linux) when the name is all numeric 
  (or always); but hiding the ".d" in the folder list.

Oh my!  Starting to visualize all the places code might be affected; unless a low-level function hides this from the main code...

[1] in my case, all msgnums were already >=2014 when I created most of my yearly archives; so I wasn't affected.

Whoa!  I should have seen collisions while _creating_ my folders....

AH-HA!!  My structure is:

MainFolderName
  Archives <== this contains only yearly directories; no messages.
    20xx
    20yy

I must have done that to reduce the sub-folders shown during expansions when I have many archive sub-folders... and fortuitously avoided this bug.

Jeroen: would that be a satisfactory solution or workaround?  Although I would suggest an error dialog:

       /!\  Error: all-numeric folder names not allowed
            in directories containing messages.

Argh! the more I think about this, the messier it gets...  my structure above is only valid as long as I never expect to include messages with conflicting names; such as via:
- drag & drop
- filtering/actions
- 'save here' compose

That would like require disallowing messages going into folders containing numeric folders....  mutually exclusive...

HTH
Comment 2 Pierre Fortin 2014-06-15 04:55:50 UTC
s/would like require/would likewise require/
Comment 3 users 2014-06-15 14:56:02 UTC
Changes related to this bug have been committed.
Please check latest Git and update the bug accordingly.
You can also get the patch from:
http://git.claws-mail.org/

++ ChangeLog	2014-06-15 14:56:02.592514083 +0200
http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=626057017bb7819152585060ff48936647a4d210
Merge: 7bc7ea0 42e1120
Date:   Sun Jun 15 14:56:02 2014 +0200
http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=42e1120c430aa5da7d3e332200a5f1060a61aafc
Author: Colin Leroy <colin@colino.net>
Date:   Sun Jun 15 14:53:31 2014 +0200

    Cache files with a dot in front when a directory with the same number exists. Fixes bug #3212, "When msgnum matches a sub-folder name, fetch fails"

http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=42eac1e8f7624533bfff86e14ae5ac007ebbe57d
Author: Colin Leroy <colin@colino.net>
Date:   Sun Jun 15 14:20:15 2014 +0200

    Fix creating a numeric directory when a message of the same number exists

http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=15784c7a971595c53c0088c80ecde92d126b23eb
Author: Colin Leroy <colin@colino.net>
Date:   Sun Jun 15 14:19:41 2014 +0200

    Fix creating a numeric directory when a message with the same number exists

http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=7bc7ea0c01b1abb6bf7ff1dddbc243011565fc59
Merge: a718abb eaf1e7a
Author: Colin Leroy <colin@colino.net>
Date:   Fri Jun 13 15:05:02 2014 +0200

    Merge branch 'master' of file:///home/git/claws
Comment 4 Jeroen Roovers 2014-06-15 15:47:52 UTC
(In reply to comment #3)
> Changes related to this bug have been committed.

But now I get a new problem.

When a cached message file, say "2014", already exists, I can add the 2014 folder to IMAP. When I then open the folder, nothing happens yet. So I copy a message to that new "2014" sub-folder and try to open that message, and yet again I can't retrieve it:

** (claws-mail:1073): WARNING **: [15:42:04] IMAP error on *****: FETCH error


(claws-mail:1073): Claws-Mail-WARNING **: can't fetch message 1


(claws-mail:1073): Claws-Mail-WARNING **: can't get message file path.


That's because a file "2014" already exists, and now the cache sub-directory "2014" cannot be created. You took care of the case of creating the file when the folder exists, but you haven't dealt with the case where the file exists and the folder is created afterwards.

Wouldn't it be better to fix this issue definitively by changing the namespace completely (and automatically invalidating and rebuilding the cache when a namespace collision occurs)?
Comment 5 Colin Leroy 2014-06-15 17:54:26 UTC
Hi,

Weird, I have handled that case in this commit:
http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=42eac1e8f7624533bfff86e14ae5ac007ebbe57d

Can you send a --debug log?

Thanks!
Comment 6 Jeroen Roovers 2014-06-15 18:04:47 UTC
(In reply to comment #5)
> Weird, I have handled that case in this commit:

I thought I had applied all the patches. Might have missed one there. I'll rebuild and try again.
Comment 7 Jeroen Roovers 2014-06-16 16:47:43 UTC
Created attachment 1406 [details]
claws-mail --debug 2>&1 | tee claws-mail.debug.log

What I did to get this output:

1) Run the command in the Description.
2) Click on the INBOX folder.
3) Click on the Gentoo-project/2013 folder.
4) Click on the last message in said folder.
5) Close claws-mail.

A cache file named '2013' already exists. In stderr this is noted as:
/home/jeroen/.claws-mail/imapcache/misha.jer-c2.orkz.net/jeroen/Gentoo-project/2013: mkdir: File exists

And from then on (step 4) things start going wrong.

This is with a git HEAD build.
Comment 8 Colin Leroy 2014-06-16 17:00:30 UTC
Ah, the folder already exists. Thanks.
Comment 9 users 2014-06-16 17:29:02 UTC
Changes related to this bug have been committed.
Please check latest Git and update the bug accordingly.
You can also get the patch from:
http://git.claws-mail.org/

++ ChangeLog	2014-06-16 17:29:02.512714931 +0200
http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=7f7a052c927a7bfcfd989c0635064ad475f7c7dd
Merge: 4451484 41d7fd7
Author: Colin Leroy <colin@colino.net>
Date:   Mon Jun 16 17:29:02 2014 +0200

    Merge branch 'master' of file:///home/git/claws

http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=41d7fd747504c2167a6bb0bb9c07b81c35302812
Author: Colin Leroy <colin@colino.net>
Date:   Mon Jun 16 17:26:59 2014 +0200

    Finish fixing bug #3212 (case where the folder exists, but is uncached
    and a mail with the same number is cached).
Comment 10 Jeroen Roovers 2014-06-16 17:50:58 UTC
(In reply to comment #9)
> Changes related to this bug have been committed.

Now it seems to behave as expected. Thanks!
Comment 11 Colin Leroy 2014-06-16 18:45:25 UTC
Thanks for confirming!
Comment 12 users 2014-06-17 20:23:02 UTC
Changes related to this bug have been committed.
Please check latest Git and update the bug accordingly.
You can also get the patch from:
http://git.claws-mail.org/

++ ChangeLog	2014-06-17 20:23:02.027859842 +0200
http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=feca0014b56965af66aec3236ccf8dbe98c0254b
Merge: 5d9fc81 4dbe7a8
Author: Colin Leroy <colin@colino.net>
Date:   Tue Jun 17 20:23:01 2014 +0200

    Merge branch 'master' of file:///home/git/claws

http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=4dbe7a8759f8928986bd00a0ea635c5c18954651
Author: Colin Leroy <colin@colino.net>
Date:   Tue Jun 17 20:21:49 2014 +0200

    Make sure we don't access out of bounds of the match string.

http://git.claws-mail.org/?p=claws.git;a=commitdiff;h=18b45d52267d0d418db258446fac2511088d9c5b
Author: Colin Leroy <colin@colino.net>
Date:   Tue Jun 17 20:07:28 2014 +0200

    Fix bug #3212, "Make wildcard address matching/autocomplete a bit smarter by sorting results in a more human way". Patch by Michael Gemlin.
Comment 13 Colin Leroy 2014-09-25 22:07:10 UTC
*** Bug 3290 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.