Bug 3682 - some senders moving from http to https for remote content; Fancy does not display this
Summary: some senders moving from http to https for remote content; Fancy does not dis...
Status: RESOLVED INVALID
Alias: None
Product: Claws Mail
Classification: Unclassified
Component: Plugins/Fancy (show other bugs)
Version: 3.14.1
Hardware: PC Linux
: P3 normal
Assignee: users
URL:
Depends on:
Blocks:
 
Reported: 2016-08-28 23:27 CEST by Pierre Fortin
Modified: 2016-09-22 13:01 CEST (History)
0 users

See Also:


Attachments
hand-crafted message illustrating the problem (2.38 KB, text/plain)
2016-09-21 21:50 CEST, Pierre Fortin
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pierre Fortin 2016-08-28 23:27:39 CEST
I've been seeing various senders changing content from http to https links.  This results in Fancy not displaying this content.

wireshark shows a connection attempt for each item (port 443):
 > SYN
 < SYN,ACK
 > ACK
 > FIN,ACK
 < FIN,ACK
 > ACK
Comment 1 Salvatore De Paolis 2016-08-29 01:58:53 CEST
Hi,
what version do you have of webkit? You can see it in the plugins window. Here I cannot reproduce your behavior. Also please can you provide off-list a email copy?

Regards
Comment 2 Pierre Fortin 2016-08-29 02:38:15 CEST
This plugin renders HTML mail using the WebKit 2.4.10 library.
By default all remote content is blocked. Options can be found in /Configuration/Preferences/Plugins/Fancy

Version: 3.14.0git2

Will send examples shortly.
Comment 3 Salvatore De Paolis 2016-08-29 05:05:21 CEST
Please check if any misconfiguration, I cannot reproduce it with either version
2.4.9 and 2.4.11
Comment 4 Pierre Fortin 2016-09-09 20:22:57 CEST
I've been digging into this as rare time permits... 
In fancy_viewer.c resource_request_starting_cb(), the 2nd last statement is:
	webkit_network_request_set_uri(request, uri);

Each time it is executed, the --debug log outputs:
** (cm:22671): CRITICAL **: void webkit_network_request_set_uri(WebKitNetworkRequest*, const gchar*): assertion 'soupURI' failed

I made the mistake of pulling(git) Webkit -- 8.5GB!

Can someone give me a clue as to what might be triggering that message?
 - version?
 - wrong call?
 - API change?
 - ...?

I've had a number of exchanges with Salvatore De Paolis; but we haven't gotten further in finding the problem.  

During our exchanges, I did notice that mouse_over on a Fancy displayed https:// link, CM displays http:// below the message pane.  I could use a patch to allow me to debug_print what is sent to webkit...
Comment 5 Salvatore De Paolis 2016-09-09 23:03:31 CEST
Hi Pierre,

try to check your libsoup version. It might be there the issue
Comment 6 Pierre Fortin 2016-09-21 21:50:16 CEST
Created attachment 1685 [details]
hand-crafted message illustrating the problem

Updating with my latest findings for the record.  
If I eventually find a solution, I'll update...

This problem is really weird... it doesn't seem to be impacted by how the message is formatted: [non-]MIME, [non-]quoted-printable, ...

This is happening on multiple sources of html mail.  It became more noticeable when one list's messages switched from http:// to https:// links.   At least, that was the initial impression.

After much digging, the problem can be illustrated by the attached email which I hand-crafted to minimize what Fancy has to deal with, and adds various combinations.

Every https:// link that Fancy attempts to get returns:
** (cm:26772): CRITICAL **: void webkit_network_request_set_uri(WebKitNetworkRequest*, const gchar*): assertion 'soupURI' failed

This issue's impact is that I have to click each link to open it in the browser; rather than give me a visual of the topic which would allow me to make instant choices as to what might be of interest, or not.
Comment 7 Michael Rasmussen 2016-09-21 22:44:11 CEST
In my opinion the root cause to the fancy problem is that the html library used is webkit 1.x where development has stopped since all new development happens in webkit 2.x. If I am not mistaken the support for CSS 3 is pure in webkit 1.x and since more and more html tools turns to CSS 3 we will see an increasing numbers of html mails which does not render correct or in worst case does not render at all.
Comment 8 Paul 2016-09-22 00:38:18 CEST
Your hand-crafted message shows no problems for me at all, and it renders just as expected, with both images (URLs) loading without problem.
Comment 9 Paul 2016-09-22 00:54:01 CEST
BTW, i'm using libsoup 2.54.1
Comment 10 Michael Rasmussen 2016-09-22 01:06:24 CEST
(In reply to comment #8)
> Your hand-crafted message shows no problems for me at all, and it renders
> just as expected, with both images (URLs) loading without problem.

Very strange because I see an empty display window when activating HTML view of the message. Running in debug mode outputs this:

mimeview.c:834:text/html
fancy_viewer.c:859:fancy_viewer_create
Vector smash protection is enabled.
fancy_viewer.c:85:fancy_get_widget: 0xfc43a70
fancy_viewer.c:85:fancy_get_widget: 0xfc43a70
openjdk version "1.8.0_102"
OpenJDK Runtime Environment (build 1.8.0_102-8u102-b14.1-2-b14)
OpenJDK 64-Bit Server VM (build 25.102-b14, mixed mode)
fancy_viewer.c:235:filename: /home/mir/.claws-mail/mimetmp/00000007.mimetmp.html
fancy_viewer.c:251:using UTF-8 charset
fancy_viewer.c:254:zoom_level: 90
fancy_viewer.c:379:navigation requested to file:///home/mir/.claws-mail/mimetmp/00000007.mimetmp.html
fancy_viewer.c:438:Starting request of 58 file:///home/mir/.claws-mail/mimetmp/00000007.mimetmp.html
fancy_viewer.c:1079:Error: Frame load was interrupted
URI: file:///home/mir/.claws-mail/mimetmp/00000007.mimetmp.html
Above file contains this:
<!--  Ruler for quoted-printable lines
0        1         2         3         4         5         6         7
123456789012345678901234567890123456789012345678901234567890123456789012345-->

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<!--
<html xmlns="http://www.w3.org/1999/xhtml"></html>
-->

<html lang="en">

<body style="margin: 0;padding: 0;background-color:
#ffffff;-webkit-text-size-adjust: none;-ms-text-size-adjust:
none;color: #505054;font-family: 'Montserrat', Arial,
sans-serif;height: 100% !important;width: 100% !important;"
bgcolor="#e5e5e5" leftmargin="0" topmargin="0" marginwidth="0"
marginheight="0">

Note: Toggle "Enable remote content" as required to clear images
      and reload them.
<br>
Warning: your browser may complain about the claws-mail.org certificate.
<p>
Image only (http -- loads)
<br>
<img
src="http://www.claws-mail.org/img/sc-bar-right.png"
height="100" width="200" border="1" alt="">
</p>

<p>
Image only (https -- does not load)
<br>
<img
src="https://www.claws-mail.org/img/sc-bar-right.png"
height="100" width="200" border="1" alt="">
</p>

<p>
Image (http -- loads) -> http://claws-mail.org
<br>
<a href="http://www.claws-mail.org">
<img
src="http://www.claws-mail.org/img/sc-bar-right.png"
height="100" width="200" border="1" alt="">
</a>
</p>

<p>
Image (https -- does not load) -> https://claws-mail.org
<br>
<a href="https://www.claws-mail.org">
<img
src="https://www.claws-mail.org/img/sc-bar-right.png"
height="100" width="200" border="1" alt="">
</a>
</p>


<!-- ################################################################ 
<p>
Mailto link -- no remote content
<a href="mailto:pf@pfortin.com" target="_blank"
style="color: #000000;-webkit-text-size-adjust:
100%;-ms-text-size-adjust: 100%;">
pf@pfortin.com</a>
</p>
-->
</body>
</html>

Some questions:
Why java output? Does fancy or webkit activate java?
Looks like CSS bug (-webkit-text-size-adjust: none; from hand crafted message): https://developer.mozilla.org/en-US/docs/Web/CSS/text-size-adjust
[1] There is a bug in old Webkit-based desktop browsers. If -webkit-text-size-adjust is explicitly set to none, Webkit-based desktop browsers, like Chrome or Safari, instead of ignoring the property, will prevent the user to zoom in or out the Web page. See Bug 56543 (affected Safari≤5 & Chrome≤26), Bug 163359, and Bug 84186.
Comment 11 Michael Rasmussen 2016-09-22 01:09:04 CEST
(In reply to comment #9)
> BTW, i'm using libsoup 2.54.1

Here:
$ dpkg -s libsoup2.4-1
Package: libsoup2.4-1
Status: install ok installed
Priority: optional
Section: libs
Installed-Size: 1049
Maintainer: Debian GNOME Maintainers <pkg-gnome-maintainers@lists.alioth.debian.org>
Architecture: amd64
Multi-Arch: same
Source: libsoup2.4
Version: 2.56.0-1
Comment 12 Michael Rasmussen 2016-09-22 01:18:15 CEST
Could also be caused by bug in certificate on claws-mail.org since the Common Name in the certificate is www.colino.net and not claws-mail.org.
Comment 13 Michael Rasmussen 2016-09-22 01:19:25 CEST
Try this URL in a browser: https://www.claws-mail.org/img/sc-bar-right.png
Comment 14 Pierre Fortin 2016-09-22 01:22:16 CEST
Thanks guys. I'm trying to get libsoup (and webkit?) updated from upcoming Mageia6 repo; though I may be in dependency hell and need to wait for Mageia6 official release which will have 2.56.0 or later.  Will keep trying. Appreciate the info.
Comment 15 Pierre Fortin 2016-09-22 01:24:28 CEST
(In reply to comment #12)
> Could also be caused by bug in certificate on claws-mail.org since the
> Common Name in the certificate is www.colino.net and not claws-mail.org.

Not likely. I get many messages from various locations; and none have reported cert issues.  Only the CM website which is why I added the warning in my crafted message...
Comment 16 Gerard Seibert 2016-09-22 13:01:35 CEST
(In reply to comment #13)
> Try this URL in a browser: https://www.claws-mail.org/img/sc-bar-right.png


I receive the following error message:

Your connection is not secure

The owner of www.claws-mail.org has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website.