https://socketlabs-1.wistia.com/medias/l7b98z6gd1?embedType=async&videoFoam=true&videoWidth=1280
The Race to Find a Top-Level Sending Domain
Trying to get the most memorable domain for your organization is becoming more difficult as millions of businesses move into the digital age. There is an especially noticeable lack of selection in quality .com domain names, as over 146 million have already been registered as of earlier this year. One of the ways this issue is partially alleviated is through the ever increasing selection of Top Level Domains (TLDs) like .net .computer, etc. You can find the full list of officially recognized TLDs by the Internet Assigned Numbers Authority (IANA) here.
Beyond the 1,588+ TLDS officially recognized by IANA, we’ve identified a market for second-level domains, also called private sub-domain registrations. These are often very short domains (such as uk.com) that resell subdomains (such as socketlabs.uk.com) in a similar fashion to how a traditional domain under a TLD (like .co.uk) is sold. In some cases, registrars are offering these second-level subdomains in-line along with traditional TLDs, which can mislead the domain buyer into seeing these choices as equal in quality.
Why Using Unofficial TLDs is Causing a Major Deliverability Issue
So, what does this have to do with email deliverability? Well, a recent change in spam patterns has been identified by the deliverability teams of Campaign Monitor and SocketLabs, impacting the ability for some legitimate senders to reach the inbox at Gmail. The issue stems from how domain reputation is calculated by Google, and the increased risks introduced when organizations select a second-level domain for their business instead of a TLD.
Spam filters operated by mailbox providers calculate a reputation for senders by analyzing prior messages and factors like the reaction of recipients. One of the primary components of sender reputation is domain reputation. This is usually the reputation of the domain that is in the From address field of the messages.
While subdomains maintain a separate reputation to a degree, mailbox providers like Google also aggregate reputation to the parent or organizational domain and bleed that reputation across all the subdomains. This “shared reputation” type of monitoring prevents a single individual domain from being a weapon with infinite subdomains that can be spun up and used to send spam. As a result, organizations that purchase second-level domains will find themselves sharing reputation with others who have also purchased a second-level domain tied to the same TLD.
Shared usage of domains in email messages isn’t actually all that uncommon, although maybe less so in the From address field. Many service providers operate shared domains for common tasks like tracking user engagement. But there is one major distinction between the shared domains operated by email service providers (ESPs) and a shared private subdomain from a registry – active abuse mitigation and vetting.
Service providers wouldn’t have a business to operate without vetting new customers to keep bad actors out and take action against the usage of their domains for improper activity. This vetting process leads to a high-quality reputation and more consistent deliverability experience for all users. Domain registrars, on the other hand, have some responsibilities to ensure domains are not used maliciously, but ultimately have little insight into the abuse activity of a given domain outside of third-party reports. So, while these registries maintain seemingly strong anti-abuse policies, the enforcement of such policies can be difficult with their limited firsthand insight, leading to shared domain sender reputation issues.
Looking Deeper into the Issue
Let’s dive into the problems email senders are facing right now using the uk.com family of subdomains as an example. Here at SocketLabs, this appears to be one of the more problematic domains. Using passive DNS data from SecurityTrails, we can see that over 100,000 subdomains exist under uk.com and have some form of recent DNS activity. While this doesn’t mean they are all sending email, it helps give an understanding of just how widespread usage of these domains can be.
In patterns seen by both SocketLabs and Campaign Monitor, mail sent using a From address with an authenticated uk.com subdomain is experiencing frequent SMTP responses from Google indicating mail cannot be accepted due to a reputation problem with the domain. Two sample bounce responses that we’ve captured here at SocketLabs are:
421-4.7.0 [ IP ADDRESS ] Our system has detected that this message is suspicious due to the very low reputation of the sending domain. To best protect our users from spam, the message has been blocked. Please visit https://support.google.com/mail/answer/188131 for more information- gsmtp
550-5.7.1 [ IP ADDRESS ] Our system has detected that this message is likely unsolicited mail. To reduce the amount of spam sent to Gmail, this message has been blocked. Please visit https://support.google.com/mail/?p=UnsolicitedMessageError for more information- gsmtp
These started occurring initially in early November 2021, but increased in frequency starting in late November 2021. At SocketLabs, we are now seeing many times more SMTP rejections (4xx) than messages accepted (250 ok) for customers sending using a uk.com subdomain in the From address field.
Seeing SMTP reputation rejections for domains that when analyzed in Google Postmaster Tools show a consistently High reputation is quite possibly one of the most interesting aspects of this whole problem. Google’s own data indicates this particular subdomain is well reputable (they also know the sending IP is highly reputable), yet they are still choosing to either delay or reject messages from it, sometimes permanently.
Travis Hazlewood and the rest of the Campaign Monitor deliverability team also investigated overall marketing performance for some of their customers experiencing the same issues. Travis noted that healthy, legitimate senders experiencing this are seeing Gmail open rates at least halved from the average at other mailbox providers and, in many cases, as low as 3-5% while other represented providers sit at 20%+, showing a clear sign of heavy filtering at Gmail due to this reputation issue. While the question of whether Google should be treating domains that act like TLDs as TLDs instead of as normal domains could/should be asked, this is reality (at least for now), leading to serious impacts for senders using them.
How Big of a Deal is This?
Just how bad is the spam problem for uk.com to be causing such a disruption for hundreds of thousands of other domains? Well, it didn’t take long diving into a Gmail spam folder to find sample messages that were downright malicious phishing and other types of generic adult dating spam that were using uk.com subdomains in the message authentication process (SPF).
Only the mailbox providers themselves are in a place to properly understand just how pervasive this issue is. With spam messages so easy to find by multiple unrelated Gmail users, Google appears to be the primary target of the spammers using these domains. Other mailbox providers like Yahoo, Outlook, and Comcast do not appear to be treating mail as harshly. This could be because they are not as heavily focused on domain reputation in their spam filtering algorithms, or because they are not receiving the volume of spam that Google is, using these domains.
It also remains unknown as to exactly why these domains are being used by spammers. One theory is that the bad actors are benefiting from the shared domain reputation that had previously been built by the thousands of legitimate senders. The spam is maybe less likely to be blocked by Google as a (new) subdomain under the uk.com domain by inheriting a somewhat good reputation — or at least that might have been the case before now. The breadth of impact to so many different private subdomains also implies the abuse is widespread, or that there have been recent changes to the Google domain reputation sharing algorithms.
This issue is so problematic for these domains that it is even impacting the organizations that manage the registration of the domains themselves. In a recent tweet by Pierre Beyssac, one of the founders of eu.org – a free private sub-domain registry – Pierre mentions that users with Gmail addresses are unable to create accounts because the verification messages use the eu.org domain in the From address field and these messages are being blocked by Google.
The moral of the story is organizations planning to use a domain to send email should choose their domains wisely. While there isn’t much that can be advised after a domain is seeing this issue outside of reporting this with the registrars, here are some general suggestions that should keep you from encountering similar issues when selecting domains in the future:
- Choose a domain from an official TLD as acknowledged by IANA
- Choose a TLD that isn’t known for spam. Spamhaus had a great tool for gauging the spamminess of a given TLD
- Be wary of domains that are extremely inexpensive or have special introductory pricing, as cost is a barrier to large scale acquisitions for malicious actors
- Follow M3AAWG (Messaging, Malware, Mobile Anti-Abuse Working Group) Best Practices for choosing your sending domain – they can be found here
This article was created and published jointly between SocketLabs’ Brian Godiksen and Campaign Monitor’s Travis Hazlewood:
