salmon.encoding module

Salmon takes the policy that email it receives is most likely complete garbage using bizarre pre-Unicode formats that are irrelevant and unnecessary in today’s modern world. These are turned into something nice and clean that a regular Python programmer can work with: Unicode.

That’s the receiving end, but on the sending end Salmon wants to make the world better by not increasing the suffering. To that end, Salmon will canonicalize all email it sends to be ascii or utf-8 (whichever is simpler and works to encode the data). It is possible to use other encodings (Salmon doesn’t live in some fictional world), but this generally frowned upon.

To accomplish these tasks, Salmon goes back to basics and assert a few simple rules on each email it receives:

1. No encoding is trusted, no language is sacred, all are suspect.

2. Python wants Unicode, it will get Unicode. In Python 3, that means str.

3. Any email that cannot become Unicode, cannot be processed by Salmon or Python.

4. Email addresses are essential to Salmon’s routing and security, and therefore will be canonicalized and properly encoded.

5. Salmon will therefore try to “upgrade” all email it receives to Unicode internally, and cleaning all email addresses.

6. It does this by decoding all codecs, and if the codec is wrong, then it will attempt to detect the codec using chardet.

7. If it can’t detect the codec, and the codec lies, then the email is bad.

8. All text bodies and attachments are then converted to Python str in the same way as the headers.

9. All other attachments are converted to raw strings as-is.

Once Salmon has done this, your Python handler can now assume that all MailRequest objects are happily Unicode enabled and ready to go. The rule is:

If it cannot be Unicode, then Python cannot work with it.

On the outgoing end (when you send a MailResponse), Salmon tries to create the email it wants to receive by canonicalizing it:

1. All email will be encoded in the simplest cleanest way possible without losing information.

2. All headers are converted to ‘ascii’, and if that doesn’t work, then ‘utf-8’.

3. All text/* attachments and bodies are converted to ascii, and if that doesn’t work, ‘utf-8’. It is possible to override this, but you’re a bad person if you do

4. All other attachments are left alone.

5. All email addresses are normalized and encoded if they have not been already.

The end result is an email that has the highest probability of not containing any obfuscation techniques, hidden characters, bad characters, improper formatting, invalid non-characterset headers, or any of the other billions of things email clients do to the world. The output rule of Salmon is:

All email is ASCII first, then encoded ASCII-safe, and if it cannot be either of those it will not be sent.

Following these simple rules, this module does the work of converting email to the canonical format and sending the canonical format. The code is probably the most complex part of Salmon since the job it does is difficult.

Test results show that Salmon can safely canonicalize most email from any written language (not just English) to the canonical form, and that if it can’t then the email is not formatted right and/or spam.

If you find an instance where this is not the case, then submit it to the project as a test case.

class salmon.encoding.ContentEncoding(base)

Bases: object

Wrapper various content encoding headers

The value of each key is returned as a tuple of a string and a dict of params. Note that changes to the params dict won’t be reflected in the underlying MailBase unless the tuple is reassigned:

>>> value = mail.content_encoding["Content-Type"]
>>> print(value)
('text/html', {'charset': 'us-ascii'})
>>> value[1]['charset'] = 'utf-8'
>>> print(mail["Content-Type"])  # unchanged
('text/html', {'charset': 'us-ascii'})
>>> mail.content_encoding["Content-Type"] = value
>>> print(mail["Content-Type"])
('text/html', {'charset': 'utf-8'})

Will raise EncodingError if you try to access a header that isn’t in CONTENT_ENCODING_KEYS

get(key, default=None)

keys()

exception salmon.encoding.EncodingError

Bases: Exception

Thrown when there is an encoding error.

class salmon.encoding.MIMEPart(type_, **params)

Bases: email.message.Message

A reimplementation of nearly everything in email.mime to be more useful for actually attaching things. Rather than one class for every type of thing you’d encode, there’s just this one, and it figures out how to encode what you ask it.

add_text(content, charset=None)

extract_payload(mail)

class salmon.encoding.MailBase(mime_part_or_headers=None, parent=None)

Bases: object

MailBase is used as the basis of salmon.mail and contains the basics of encoding an email. You actually can do all your email processing with this class, but it’s more raw.

append_header(key, value)

Like __set_item__, but won’t replace header values

attach_file(filename, data, ctype, disposition)

A file attachment is a raw attachment with a disposition that indicates the file name.

attach_text(data, ctype)

This attaches a simpler text encoded part, which doesn’t have a filename.

property body

get_all(key)

items()

keys()

Returns header keys.

walk()

salmon.encoding.VALUE_IS_EMAIL_ADDRESS(v)

salmon.encoding.apply_charset_to_header(charset, encoding, data)

Given a charset and encoding, decode data into unicode, e.g.

>>> print(apply_charset_to_header("utf-8", "Q", "=142ukasz"))
łukasz

encoding is case insensitive and must be one of B or Q

salmon.encoding.attempt_decoding(charset, dec)

Attempts to decode bytes into unicode, calls guess_encoding_and_decode if the given charset is wrong.

salmon.encoding.from_file(fileobj)

Reads an email and cleans it up to make a MailBase.

salmon.encoding.from_message(message, parent=None)

Given a MIMEBase or similar Python email API message object, this will canonicalize it and give you back a pristine MailBase. If it can’t then it raises a EncodingError.

salmon.encoding.from_string(data)

Takes a string, and tries to clean it up into a clean MailBase.

salmon.encoding.guess_encoding_and_decode(original, data, errors='strict')

salmon.encoding.header_from_mime_encoding(header)

salmon.encoding.header_to_mime_encoding(value, not_email=False)

salmon.encoding.normalize_header(header)

salmon.encoding.parse_parameter_header(message, header)

salmon.encoding.properly_decode_header(header)

Decodes headers from their ASCII-safe representation

salmon.encoding.properly_encode_header(value, encoder, not_email)

The only thing special (weird) about this function is that it tries to do a fast check to see if the header value has an email address in it. Since random headers could have an email address, and email addresses have weird special formatting rules, we have to check for it.

Normally this works fine, but in Librelist, we need to “obfuscate” email addresses by changing the ‘@’ to ‘-AT-’. This is where VALUE_IS_EMAIL_ADDRESS exists. It’s a simple lambda returning True/False to check if a header value has an email address. If you need to make this check different, then change this.

salmon.encoding.to_file(mail, fileobj)

Writes a canonicalized message to the given file.

salmon.encoding.to_message(mail)

Given a MailBase message, this will construct a MIMEPart that is canonicalized for use with the Python email API.

N.B. this changes the original email.message.Message

salmon.encoding.to_string(mail, envelope_header=False)

Returns a canonicalized email string you can use to send or store somewhere.