Bug 4075 - Replies include only the first paragraph of the original message
Summary: Replies include only the first paragraph of the original message
Status: RESOLVED FIXED
Alias: None
Product: Claws Mail (GTK 2)
Classification: Unclassified
Component: UI/Compose Window (show other bugs)
Version: 3.17.0
Hardware: PC Mac OS X 10.2
: P3 major
Assignee: users
URL:
Depends on:
Blocks:
 
Reported: 2018-08-22 13:51 UTC by Perry E. Metzger
Modified: 2018-08-24 23:52 UTC (History)
0 users

See Also:


Attachments

Description Perry E. Metzger 2018-08-22 13:51:56 UTC
(MacOS High Sierra 10.13 isn't included in the OS list but that's what I'm on.)

If I reply to a message under 3.16, the entire original message is included in the reply. If I reply under 3.17, only the first paragraph is carried over. This makes Claws almost unusable for me because I can't actually reply paragraph by paragraph to a long email that I've been sent. Ideas on fixing this actively solicited. I'm happy to provide screen grabs to show the problem, dumps of my config files, etc.
Comment 1 Paul 2018-08-22 14:09:43 UTC
What happens if you select all the message text before hitting reply?

Are you certain that you are not selecting just the first paragraph?
Comment 2 Perry E. Metzger 2018-08-22 14:55:40 UTC
If I select all the message text, I still only get the first paragraph. If I select three paragraphs in the middle of the message, I only get the first of the three selected paragraphs, so it's clear selection is having an effect, but not the effect expected.

And I'm absolutely sure I'm not selecting anything under normal circumstances.
Comment 3 Perry E. Metzger 2018-08-22 14:59:18 UTC
I performed an experiment. It appears that the reply ends after the first blank line after the first text. Initial blank lines don't stop the first paragraph from being found.

Lines containing only spaces don't cause the "cut" behavior, so if there's an initial paragraph followed by a line containing two spaces followed by more text, both the initial paragraph and the following text will be in the reply. It seems to strictly stop copying at the first truly blank line after the first text in the message.
Comment 4 wwp 2018-08-22 15:02:43 UTC
Is there a signature separator somewhere in between the paragraphs?
Did you define any custom signature separator in your accounts or in global prefs?
Comment 5 Perry E. Metzger 2018-08-22 15:30:44 UTC
"Is there a signature separator somewhere in between the paragraphs?" -- no.

"Did you define any custom signature separator in your accounts or in global prefs?" -- I have not edited the signature separator settings at all, ever. That said, it is not impossible that the upgrade somehow corrupted that setting or others. I will check.
Comment 6 Perry E. Metzger 2018-08-22 15:37:09 UTC
I can find no signature separator setting globally. For my default account, it appears to be normal (that is, "--").

BTW, let me be clear: I've been programming for about 40 years. I hack on compilers and device drivers and the like. I'm also the port maintainer for claws on the mac and I'm in the middle of fixing up the port of claws to the native aqua version of gtk for the mac. I do know what I'm doing. I wouldn't have created a trouble ticket otherwise. That doesn't mean I haven't done something stupid, but it does mean you can presume I've tried checking obvious things.
Comment 7 Perry E. Metzger 2018-08-22 15:38:54 UTC
Is there any easy way to check for corruption in the files containing my preferences?
Comment 8 wwp 2018-08-22 15:50:08 UTC
Eye-looking at the file contents or running Claws Mail..
But I wonder why you think your config file(s) could be corrupted? By what exactly?
Comment 9 Perry E. Metzger 2018-08-22 15:52:27 UTC
I wonder about corruption because you asked if the signature separator had changed. Since I haven't changed it, the only other thing that could have changed it would have been some sort of process beyond my control. As for what might have done this, I don't know, it isn't even like the odds of any sort of corruption are high. I have no particular evidence that this has happened, but I'm trying to debug a bad problem, and excluding possibilities is important.
Comment 10 Paul 2018-08-22 15:56:59 UTC
Can you either send me a copy of a message file which displays such behaviour (using 'Forward as Attachment'), or attach a message file here?
Comment 11 wwp 2018-08-22 15:58:19 UTC
We may have other technical considerations that might involve signature separators w/o them being broken on your side, anyway (it's more about a change in the way they are handled internally, but it's just a possibility at the moment).
Just curious, what are the mail programs used to send emails you're attempting to reply when you face the issue?
Comment 12 Perry E. Metzger 2018-08-22 16:00:47 UTC
I've examined the contents of my preferences files in .claws-mail/*rc -- I see no evidence of anything amiss. Checking separators in the accountrc file reveals the normal "--" separator.
Comment 13 Perry E. Metzger 2018-08-22 16:02:36 UTC
ALL mail messages, if replied to, display this behavior. I could send you a mail message that displays the behavior, but as all of them display it, I don't think it has to do with the message. Anything you write to me would have exactly the same behavior. If you sent me a mail message containing:

"
line one

line two
"

only "line one" would appear quoted if I hit reply.

I can reproduce the problem with every single message in my mailbox.
Comment 14 Paul 2018-08-22 16:04:44 UTC
(In reply to comment #12)
> Checking separators in the accountrc file
> reveals the normal "--" separator.

Just FYI, 'normal' is "-- "
Comment 15 Perry E. Metzger 2018-08-22 16:08:21 UTC
BTW, feel free to send me an email and I'll happily reply with a screenshot of what happens when I try to reply to that email.

"Just FYI, 'normal' is "-- ""

$ fgrep sep accountrc | hexdump -c
0000000   s   i   g   n   a   t   u   r   e   _   s   e   p   a   r   a
0000010   t   o   r   =   -   -      \n   s   i   g   n   a   t   u   r
0000020   e   _   s   e   p   a   r   a   t   o   r   =   -   -      \n
0000030   s   i   g   n   a   t   u   r   e   _   s   e   p   a   r   a
0000040   t   o   r   =   -   -      \n   s   i   g   n   a   t   u   r
0000050   e   _   s   e   p   a   r   a   t   o   r   =   -   -      \n
0000060   s   i   g   n   a   t   u   r   e   _   s   e   p   a   r   a
0000070   t   o   r   =   -   -      \n   s   i   g   n   a   t   u   r
0000080   e   _   s   e   p   a   r   a   t   o   r   =   -   -      \n
0000090
Comment 16 wwp 2018-08-22 16:09:34 UTC
Q: what's you Claws Mail package distributor? Are you able to build from the sources in Git repo?
Comment 17 wwp 2018-08-22 16:10:41 UTC
BTW, your signature seps look OK.
<off-topic>
It's true that the conventional sep is "-- " but since the trailing " " is not visible, many people think the sep is "--"
</off-topic>
Comment 18 Perry E. Metzger 2018-08-22 16:24:53 UTC
(In reply to comment #16)
> Q: what's you Claws Mail package distributor?

MacPorts, and I'm the port maintainer, so I can't complain to anyone but me.

> Are you able to build from the sources in Git repo?

I could if you need me to. I can also try building with any patches you wish to give me.
Comment 19 wwp 2018-08-22 16:40:51 UTC
Oh, good to know..
Then I'd ask you to build from the latest git revision, if you're pleased too. There's been a commit addressing a potentially connected issue, few times after the 3.17.0 release, and we'd like to know if that commit is fixing or not on your side.
Comment 20 Perry E. Metzger 2018-08-22 16:50:02 UTC
Building out of the master branch in git fixes the problem.

Do you know what the issue is? And can we get a 3.17.1 with a fix?
Comment 21 Perry E. Metzger 2018-08-22 16:53:00 UTC
Oh, and side comment: the git master version identifies itself as 3.17.0 when I build it. It might be nice if it had some indication that it was a non-release version, if only to prevent confusion. (I worried for a moment that I'd started the wrong copy.)
Comment 22 wwp 2018-08-22 17:08:01 UTC
The issue was a wrong printf format used ("%\n") during signature separator manipulations, this causes a crash w/ some musl library, obvisouly a wrong behaviour w/ MacOS's libSystem, and nothing at all w/ some glibc versions.

I can't tell you about a 3.17.1 to come yet, nor how to get an visual indication that you're building from a non-release copy of the sources (here it shows 3.17.0git9), but I remember somebody reported the same as you recently.
Comment 23 wwp 2018-08-22 17:09:40 UTC
And BTW, thanks for reporting, testing and helping in narrowing down what the issue was!
Comment 24 Perry E. Metzger 2018-08-22 17:11:48 UTC
Could you direct me to the git commit ID that fixes it so I can add the patch to the MacPorts build? I imagine it's a one-liner...
Comment 25 Perry E. Metzger 2018-08-22 22:42:16 UTC
Is the commit e0a319b4b672c592f9824509d948914a4d167a1e ?
Comment 26 Perry E. Metzger 2018-08-22 22:56:18 UTC
Hrm. Adding that patch didn't seem to fix the problem, at least not on its own.
Comment 27 Paul 2018-08-23 11:58:09 UTC
(In reply to comment #26)
> Hrm. Adding that patch didn't seem to fix the problem, at least not on its
> own.

Can you test builds with that patch and others, in order to determine what other patch/commit (or patches/commits) is needed?
Comment 28 Paul 2018-08-23 11:59:35 UTC
(In reply to comment #27)
> (In reply to comment #26)
> > Hrm. Adding that patch didn't seem to fix the problem, at least not on its
> > own.
> 
> Can you test builds with that patch and others, in order to determine what
> other patch/commit (or patches/commits) is needed?

Try this one additionally first: 6406496b93866bb472a221ba93422fdb56c95773
Comment 29 Perry E. Metzger 2018-08-23 14:37:58 UTC
(In reply to comment #28)
> (In reply to comment #27)
> > (In reply to comment #26)
> > > Hrm. Adding that patch didn't seem to fix the problem, at least not on its
> > > own.
> > 
> > Can you test builds with that patch and others, in order to determine what
> > other patch/commit (or patches/commits) is needed?
> 
> Try this one additionally first: 6406496b93866bb472a221ba93422fdb56c95773

I added it. It didn't seem to fix things. If this is really surprising, I can double-check that I am running the correct version, but I'm fairly sure. Any others I should try?
Comment 30 Paul 2018-08-23 17:45:12 UTC
(In reply to comment #29)
> Any others I should try?

Yes. There are only are a few commits since 3.17.0 at this time. Please work your way through them, testing as you go.
Comment 31 Perry E. Metzger 2018-08-24 22:26:27 UTC
Okay, it turns out only e0a319b4b672c592f9824509d948914a4d167a1e is all that was needed.

The reason I was deceived was this:

What's in git and what's in the release aren't the same. In particular, that commit patches a yacc file, and the outputs of running bison over that are in the release (but not in the git tree), _and_ the Makefile doesn't properly note the dependency between the .c file generated and the .y file.

Thus, I added that patch on top of the MacPorts build and the .y file wasn't reprocessed, and thus I didn't see the fix.

This took a ridiculous amount of time to figure out. Please fix the dependencies in the makefile.

Anyway, having found this, I'm not sure what we do with the bug report. It can't exactly be closed because the problem is still present in the latest release, but it is indeed fixed in the git repo master.
Comment 32 Andrej Kacian 2018-08-24 22:40:48 UTC
(In reply to comment #31)
> What's in git and what's in the release aren't the same. In particular, that
> commit patches a yacc file, and the outputs of running bison over that are
> in the release (but not in the git tree), _and_ the Makefile doesn't
> properly note the dependency between the .c file generated and the .y file.
> 
> Thus, I added that patch on top of the MacPorts build and the .y file wasn't
> reprocessed, and thus I didn't see the fix.
> 
> This took a ridiculous amount of time to figure out. Please fix the
> dependencies in the makefile.

Alas, not possible when working with a git checkout, unless you enable the maintainer mode. See the bit about rebuild rules here: https://www.gnu.org/software/automake/manual/html_node/maintainer_002dmode.html#maintainer_002dmode

Note You need to log in before you can comment on or make changes to this bug.