Email Analytics that Drive Performance

While many companies think that they can sit back and relax once they’ve selected a reliable email services provider (ESP), the reality is quite different. Choosing an ESP with rock-solid infrastructure, robust sending technology, and intelligent delivery strategies is certainly a critical first step. But beyond these foundational issues, there are equally critical email performance factors that are dependent on the email sender and that business’ adherence to email best practices.

The key revelation for many high-volume email businesses is that performance is a function of a multitude of other qualitative and quantitative factors which combine to influence the willingness of mailbox providers to accept and deliver mail to an inbox. Further, even after a message successfully reaches a recipient’s inbox, there are important user engagement considerations that that must be mastered in order to build and maintain a strong domain reputation that is well-respected by mailbox providers over time. Collectively, these concerns should be thought of as a set of constantly changing rules that email senders must honor, or else they face a downward spiral of poor performance statistics and sender reputation woes.

The silver lining in this story is that senders in complete control of their own destiny because they are in complete control of the non-infrastructure variables that are so important. For the ease of discussion, these variables can be condensed into three basic categories:

1. the quality of the recipient list
2. the construction of the email content
3. the relevance of the message to the recipient audience.

As these categories illustrate, the key business driver for all email senders is simply the choice of “who” they send messages to and “what” types of messages they send. It is overwhelmingly these choices that determine how effective and successful email communications will be. However, because the rules defined by mailbox providers and the algorithm’s automatically judge incoming emails are constantly changing, it is essential that sender evaluate more than just typical performance tracking statistics.

Understanding StreamScore™ Components

Spam complaints are the most commonly mentioned and understood issue for most people, but they are only a single point of measurement about a stream of email messages. A full analysis of the messages including looking at type (marketing vs. transactional vs. person-to-person), content, volume, failure rate, opens, clicks, and other factors is required to determine what what’s really happening. These data points are very real from a standpoint sender reputation and the cold, data-driven world of email deliverability rules. Regardless of your intent, if your mail messages are scoring poorly, then your current mail and future mail will be more likely to fail and your ROI will suffer.

Below is a description of each of the StreamScore™ components and an explanation of why each is relevant. These components are:

• Hard Failure Score
• Block Score
• Spam Score
• Complaint Score
• Engagement

Hard Failure Score

Hard failures are a direct reflection of the quality of a mailing list. Specifically, these failures are situations where the receiving mailbox rejects the incoming message because the address information is invalid, outdated, or the message fails a filter?). A high percentage of hard failures is typically an indication that the sending organization is not following best practices with regard to list acquisition, hygiene, or management.

SocketLabs’ Hard Failure Score metric is based on the rate or hard failures that are observed for any given number of sent messages. For transactional mailings, the hard failure score is typically high. This is because transactional recipient lists are composed primarily of active customers who have a clear business incentive to provide accurate address information and they tend to keep their address information updated over the course of their relationship with the sender. By contrast, marketing lists and older client lists are more likely to contain address errors that lead to hard failures. Purchased lists or second-hand lists (those not acquired created directly by the sending organization) that are not opt-in are notorious for having outdated names, which is why these types of lists are disallowed by most reputable ESPs, including SocketLabs.

Blocked Message Score

The Block Score is based on the Block Rate which is SocketLabs’ analysis of the failed and bounced messages that are rejected due to content-related issues. The term content in this context refers to the entire email header and message. The analysis looks at the error codes returned by the recipient mailboxes to look for responses that explain why they chose to not to deliver the message to the addressee’s inbox.  The block score is based on the rate of “content” driven failures that are observed over a period of time.

Since each mailbox provider uses a different set of content filters and establishes their own set of rules, there are a multitude of reasons why messages can be blocked. In general messages can fail because of what they do contain, or what they don’t. For example, a message that contains “spammy” looking messages regarding free offers or promotions may be rejected. Similarly, a message may be rejected if it does not contain sufficient authentication credential such as SPF or DKIM. Typically, less than 1% of failures are content related, so organizations experiencing block rates greater than this will see their Block Score negatively impacted. Once identified, issues causing blocks can be often be corrected.

Spam Score

The Spam Score is an estimate of the degree to which a senders’ messages are being classified as spam by receiving mailbox systems. SocketLabs calculates an approximate spam rate via a proprietary methodology that evaluates samples each client’s mail streams.

The underlying factors driving a Spam classification are very similar in concept to those driving the Block Score. Each looks at the message content to make a determination of a message’s worthiness. The difference between the two is really how the receiving mailbox chooses to handle it. In the case of a block, the message is rejected by the mailbox provider and notification is sent back to the sender. In the case of spam, the messages is ‘accepted’ by the mailbox, but it is placed into the spam folder without generating a response back to the sender. Each mailbox provider chooses their own methodology and filtering strategy by which to make this determination and can judge the same exact message differently. Beyond looking just at the message content, mailboxes are very sensitive to a senders’ domain reputation. As a result, if a sender is continually using bad email practices or they had an email account somewhere get suspended, they may develop a poor domain reputation that can drive mail streams to be placed in a recipient’s spam or junk folder.

Complaint Score

The Complaint Score is calculated based on the number of complaints generated by your mail streams. Data is directly available from certain mailbox providers regarding the complaint rate, however many mailbox providers, including Google, do not share this information. Data that contributes to SocketLabs’ Complaint Score calculation is based on data received back from Yahoo! and Microsoft. Complaint data is available only when B-to-C communication is taking place, so many clients, especially those that send exclusively transactional mail, will not generate complaint data. If no complaint data is available, this component is removed from the StreamScore™ calculation.

For accounts where complaint rates are measurable, the ability to achieve a high score is determined by two interrelated factors: a) the content of the outbound message, and b) the frequency with which those messages are being sent. The message content affects the frequency because recipients are only willing to tolerate a certain message a certain number of times before they become annoyed and being complaining. For example, if you are sending out a coupon to your entire list every day, this message/frequency combination is very likely going to generate complaints. If, however, you are sending important alert messages – alerts which are deemed important and valuable by the recipient – a daily frequency may be perfectly ok. The bottom line is that your approach should be thoughtful and appropriate so as to not to annoy people – it’s that simple.

The Importance of Engagement

Mailbox providers understand the value of engagement data for giving a more personalized experience to users and for helping to fight off spammers. They give significant weight to strong open and click rates in the email filtering process because the recipients themselves – the most important judges of quality, relevance and value – are saying that the messages are desirable. As such, strong engagement can compensate for and overcome other possibly negative sender reputation metrics like spam complaints, hard failures, and message content concerns.

SocketLabs offers engagement tracking for opens, clicks and capabilities on all of our plans. This feature is optional, and the Engagement Score is only calculated for clients who choose to configure and use it. Due to the tremendous value that engagement data provides, it is highly recommended.

Read the StreamScore™ Data Analysis blogs or visit the StreamScore™ homepage.