X-Message-Number: 10587
Date: Wed, 14 Oct 1998 01:43:52 -0400 (EDT)
From: "Kevin Q. Brown" <>
Subject: Administrivia: Ugly Email

This message is about the form, not the substance, of CryoNet
messages.  Here is the problem:

    An increasing number of CryoNet messages are posted in
    formats that make the digest hard to read.  Traditionally
    this has been a result of new users who have not yet trained
    their mail programs to behave properly, but now it is due
    mainly to the new, MIME- and web- aware email/browser packages
    that seem to be losing the capability of generating simple
    ASCII messages.

The types of problems I have seen (not all on CryoNet) include:

    (1) Word processor ASCII dump - one long line for each paragraph.
    (2) Email composed with a variable width font (not a fixed width
        font such as Courier), which usually results in lines longer
	than 80 characters.
    (3) Replies that contain the _entire_ previous digest.
    (4) Replies (such as unsubscribe requests) that are sent to the
        CryoNet posting address rather than "".
	Fortunately, the software usually catches these automatically,
	but people who misspell "unsubscribe" in creative ways can
	elude that filter.
    (5) Overly long messages.  (It is much nicer to give a URL to a
        large document rather than blasting the entire contents directly
	to everyone on the mailing list.)  Currently, CryoNet limits
	each individual message to 40 KB (including the email headers).
    (6) Base64-encoded MIME attachments (usually images, binary
        executables, MS-Word documents, or even TNEF files).
	These are not appropriate for a text-based digest, so they
	are truncated automatically from CryoNet messages.
    (7) "Quoted-Printable" MIME attachments.  These currently are the
        worst offenders.  They consist mostly of readable ASCII, but
	include funny control codes containing equal signs that uglify
	the messages.  Worse, the latest variety (from Outlook Express)
	confound my hack for repairing CompuServe email lines that have
	been wrapped around prematurely, and the result is a message
	with lines that are both long and illegible.
    (8) Appended HTML copies of the ASCII text.  Fortunately, these
        seem to be universally redundant, so no information is lost by
	truncating the messages automatically when HTML attachments
	are detected.
    (9) VCARD "signature" MIME attachments from Netscape mailers.
   (10) Weird, non-printable, or hyperascii characters in messages,
        which look incomprehensible, terminate digests prematurely on
	DOS machines, or make some US ASCII - based mail readers think
	that the messages are binary code.  I now have a simple and
	effective filter for these characters.  My efforts have not
	always succeeded, though.  My first filter for the DOS End-Of-File
	character Control-Z (Decimal 26, Hexadecimal 1A) accidentally
	filtered out all instances of the number "26" from messages!
	See Message #5210 "CryoNet Feature #26" for the full story.

Possible solutions to the badly formatted message problem include:

    (1) Ignore it and live with it.
    (2) Automatically reject messages that are formatted badly.
    (3) Automatically reformat messages that are formatted badly.
    (4) Change the CryoNet digest format to a set of MIME attachments.

I don't like solution (1), solution (2) seems inhospitable (except
for obviously erroneous, voluminous, or redundant messages),
solution (3) may be difficult to do properly, and solution (4) would
make CryoNet digests indigestible by traditional mail readers.  That
leaves solution (3) as the method of choice.

Reformatting "Quoted-Printable" attachments seems to be the most
important task at this time, so, unless I hear strenuous complaints,
I'll install software that (A) detects "Quoted-Printable" attachments
and (B) reformats them to fit into 75 columns without any of the
control codes that uglify those messages.  Hopefully this software
will resolve the "Quoted-Printable" problem without introducing new
and exciting problems of its own.

    Kevin Q. Brown
    

Rate This Message: http://www.cryonet.org/cgi-bin/rate.cgi?msg=10587