Created attachment 879 [details] Suggested patch fixing base64 decode Previously reported in http://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=2076#c12 In claws-mail-3.7.6, decoding base64 (attachment) in claws-mail has some bugs. The attached patch fixes the problems at least for my use cases. The changes: + In base64_decoder_decode(...) * An 'inlen' parameter added. * Checking for padding character '=' changed. It is now also used as sufficient (though non-necessary) condition for end-of-input. * "Unrecognized" characters like 'null' or 'space' or silently ignored, and do not imply end-of-input. * Comments about saved tail of input were added. + In procmime_decode_content(...) [the ENC_BASE64 case] * The input chunks are read by fread(...) (instead of fgets(...)). This solves the problem of reading just one line at a time. Similar change should probably be applied to other cases (such as ENC_QUOTED_PRINTABLE, ENC_X_UUENCODE, default), but I did not have cases to test them with. * Check is added that the actual number of desired chars are read.
And do you have any example of those use cases?
I will give such an example beginning next week.
Created attachment 883 [details] semi example The 'real' example that I have may contain confidential data. I still hope to give similar full example. Meanwhile I attach a "sub-msg" where I took just part of an original example and changed some text segments. But the actual problem - of split lines with spaces where base64-decoding of claws-mail fail - is there.
Hi, I've looked at the patch and at first sight it looks OK. However, I also looked at the example email and it's not MIME compliant. Half-lenght lines are suspiscious, lines starting with space are wrong, and each line is supposed to be parseable without the previous or next one. I'm usually in favor of being strict in what we send and tolerant in what we get, but in such cases I'm not, because MIME parsing is very complicated and working around other software bugs only caused us problems, historically. Please reopen only if you can find a problem with mime-compliant emails.
Hello Colin, You must be more knowledgeable about MIME-related standards and RFCs. Therefore, I do not want to simply re-open this bug and start an endless close-open-close sequence. But please consider the following two observations I have. (1) The base64-encoded attachments I am experiencing at my work place (again sorry for not being able to send them as-is), can be nicely extracted by Mozilla-Thunderbird, GMail-Web-interface, and probably other clients. And as I previously mentioned, claws-mail without my suggested patch fail to decode them. (2) I dived to a specific software-code sending these encoded attachments. I noticed that the encoded base64-buffer is sent as one huge line. But our mail server Postfix, folds it into 990 characters sized lines. Indeed Postfix documentation http://www.postfix.org/postconf.5.html#smtp_line_length_limit says: smtp_line_length_limit (default: 990) The maximal length of message header and body lines that Postfix will send via SMTP. Longer lines are broken by inserting "<CR><LF><SPACE>". This minimizes the damage to MIME formatted mail. By default, the line length is limited to 990 characters, because some server implementations cannot receive mail with long lines. So the popular Postfix is source of 'lines starting with space'. regards -- yotam
Hi, I've read RFC 2045 again to make sure I wasn't mistaken about these line lengths, and it indeed says "The encoded output stream must be represented in lines of no more than 76 characters each". But just then, it also says "All line breaks or other characters not found in Table 1 must be ignored by decoding software. In base64 data, characters other than those in Table 1, line breaks, and other white space probably indicate a transmission error, about which a warning message or even a message rejection might be appropriate under some circumstances". An important detail indeed. As we already have a warning message if decoding fails, I think your patch fits, so I'll test it and commit it. Thanks!
Changes related to this bug have been committed. Please check latest CVS and update the bug accordingly. You can also get the patch from: http://www.colino.net/claws-mail/ 2010-08-25 [colin] 3.7.6cvs31 * src/procmime.c * AUTHORS * src/gtk/authors.h * src/common/base64.c * src/common/base64.h Fix bug #2245, "base64 decoding fails with line breaks" Patch by Yotam Medini
I am using Clam Mail 3.7.8 under Gentoo Linux and base64 decoding does not work correctly for me, see attached mail.
Created attachment 958 [details] Source of base64-encoded mail
Created attachment 959 [details] Mail displayed by Claws
Hi, It works for me, can you check with normal CFLAGS if you use folkoric ones?
Ah, forgot to say I am on armv7: CFLAGS="-O2 -pipe -mcpu=cortex-a9 -mfpu=vfpv3-d16 -mfloat-abi=softfp" CHOST="armv7a-unknown-linux-gnueabi" gcc-4.5.2, glibc-2.11.3 I think this CLFAGS are quite safe but maybe I am wrong? Any suggestions on how to debug this?
Oh, on ARM. It's probably another bug, then, but still related to endianness... Your gcc is very recent. I don't know how to help you debug that...
Can you check the output when running claws-mail --debug ?
I recompiled claws-mail but currently it does not start anymore.. but I will check with debug-option soon. Thanks for your quick response and the hints!
Colin, I have access to ARM boxes if you need to test something. They're not with Gentoo but with Debian GNU/Linux, but that's not relevant to this bug (I believe to have observed it also in Debian).
Ok.. segfault on startup seems to be another bug, apparently related to startup-notification, I will take a look at that later. So, after disabling startup-notification I managed to start with debug-option and this is the output when clicking on the base64-message mentioned above: imap.c:1410:trying to fetch cached /home/fabian/.claws-mail/imapcache/[...] imap.c:1419:message 7441 has been already fully cached. message/rfc822 (offset:0 length:3975 encoding: 6) text/plain (offset:3399 length:576 encoding: 4) textview.c:655:TIMING textview_add_part : 0s008ms textview.c:1058:Viewing text content of type: plain (length: 430) textview.c:758:TIMING textview_add_part : 0s138ms textview.c:775:TIMING recursive_add_parts : 0s138ms textview.c:821:TIMING recursive_add_parts : 0s146ms textview.c:583:TIMING textview_show_part : 0s147ms summaryview.c:3505:TIMING summary_display_msg_full : 0s289ms I can not see any suspicious messsage in there.
Recompiled with CFLAGS="-march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp" but still base64 decoding does not work properly. Does also not work for attachments, btw. So the bug seems to be common to ARM systems and not related to my specific CFLAGS.
The bug is fixed for me with version 3.7.9! Thanks :)