Research: What Do Bounces Really Mean for Super Senders?

Bounces are inevitable. Every email you send is almost assured to see instances where that mail cannot be delivered — whether temporary or permanent. These bounces create a variety of responses from marketers: some ignore them, some meticulously analyze bounce logs, and others retry sending every. single. one. The recommended approach is somewhere in the middle, leaving many marketers to question how they should handle certain types of bounces and what those rejections really mean.

Methodology

Given all of the interest in how senders should react to bounces, two of our data-minded experts set out to find the answers. Covering a data set of nearly 20 billion attempted deliveries, we analyzed the likelihood that a recipient would re-engage with a marketing brand after a bounce was registered to that address. For this analysis, we disregarded transactional messages since those may not have been in the same vein of communication (e.g. password resets versus marketing messages).

Like most ESPs, MessageGears groups bounces into specific categories depending on the response received from the server where we attempt delivery. A few of the most common include category 10, which indicates the recipient address is invalid, category 22 for mailbox full/over quota, and categories 51 and 52 for spam blocks. Learn more about block lists. 

As a quick reminder, MessageGears doesn’t store any personally identifiable information – all of that stays on premises in our users’ secure data warehouse. For this analysis, we used pseudonomynized records in accordance with GDPR and CCPA.

NIck Ziech-Lopez, Senior Product Owner of Cloud Operations:

Brad, I see a lot of our users asking similar questions about bounce types: Which bounces mean a message should be sent again, and which bounces mean the mailbox is likely invalid or unused? I see we have a few different categories of bounces here, so let’s start with an easy one: mailbox full. If a Super Sender receives a bounce because a recipient’s mailbox is full, should they try again?

Brad Gurley, Director of Deliverability:

One of the most frequently debated topics in email pertains to mailbox full bounces. In the early days of email (also known as B.G.: Before Gmail), these bounces were very common. Most webmail and hosted servers had very limited space available for email storage (2-4MB on average) prior to the launch of Gmail and its 1GB of free storage. Over time, Gmail’s free storage grew, and other providers increased their own storage limits to keep up. As a result, free email services now regularly offer at least 2-5GB of storage space for emails. Compound that with the fact that around half of all emails are read on a mobile device (as opposed to checking every day or two on a computer), and it’s easy to see why inboxes are less likely to be full than they once were.

When it comes to these bounces, some experts recommend suppressing recipients after a “mailbox full” while others recommend leaving them in the sending rotation. To figure it out, let’s ask one of our favorite questions: what does the data say?

Mailbox Full

Recipients who received a “mailbox full” bounce had better than an 8% chance of opening a subsequent message within a week of that bounce. Spread the timeframe to one year, and the chances increase to over 30%. This tells us that “mailbox full” is far from a death sentence for an email address, and it’s probably not a good idea to suppress a recipient after a single occurrence.

With that said, keep in mind that two-thirds of these recipients didn’t open another message within a year, so many are still likely to be abandoned or rarely used addresses. We’d recommend suppressing if you see repeated mailbox full bounces without a successful delivery over the course of 3-4 weeks.

NICK: Wow, I had no idea that many people were constantly dealing with full mailboxes. Moving on, I see a large amount of users with a Timeout bounce end up eventually re-engaging. Why is that?

BRAD: Delivery Timeout bounces often occur when the receiving server has rate limits in place to prevent mail from being delivered too quickly. Eventually, the delays exceed the allotted expiration for the delivery attempt and the delivery times out.

In many cases, these rate limits are dynamic and can depend on the reputation of the sender or how recipients are engaging with the messages. Because of that link to reputation and engagement, we expect to see relatively high rates of delivery after these bounces — and the data doesn’t disappoint.

Delivery Timeouts

Here, we see over 40% of these bounced recipients opening a message within a week of a delivery timeout, and over 12 months that number jumps to over 80%. This data makes it clear that delivery timeouts should almost always be retried.

NICK: OK, I see … since a rate limit doesn’t necessarily mean bad behavior and could be just an ISP setting, many of those recipients could be looking forward to those messages and be actively wanting to engage. But let’s look on the flip side and examine bad behavior. Can you tell me about spam?

BRAD: An unfortunate truth is that many bounces are caused by mailbox providers classifying the mailing as spam. However, not all spam designations are created equal — MessageGears classifies two types of spam bounces: Spam Block (51) and Spam Content (52).

Bounces categorized as Spam Block typically reference sender reputation. The bounce detail may specifically identify the IP address or domain used to send the message or could be a generic response indicating the message is believed to be spam. Bounces in the Spam Content category will usually have specific references to content in the bounce response. These generally also include messages with blocklisted URLs contained in the body content (image hosting, click tracking, etc).

Spam Bounce

This data shows us an interesting dichotomy: mail classified generically as spam sees a consistently higher likelihood of future engagement than mail specifically called out for spam content. It’s impossible to pinpoint a cause with this data, but my first take would be that it’s often hard to identify specifically which content in the message is causing the bounce. Bounces for a bad reputation often tell you whether it’s the domain or IP that needs to be addressed, and senders can take action based on that determination. Content bounces rarely identify specifics, sometimes leaving senders to a process of trial-and-error to find the specific cause.

It’s also possible that these content-specific bounces are generated by filters that are more strict than the average recipient (corporate or institutional domains, for example). Recipients at domains with these types of filters may be less likely to receive the messages in the future because of the elevated filtering criteria.

NICK: Interesting that not all spam bounce designations are created equal. But before we go, I always hear of the designation of ‘hard’ versus ‘soft’ bounces, and that DNS failures and invalid recipients are typically hard bounces. Can you tell me what that means, and does the data back it up?

BRAD: Both of those are classified as ‘hard’ bounces, but a better term might be ‘permanent’ bounces to indicate they should be suppressed from future sends. In the case of DNS failure, our mail servers attempt to locate the domain of the recipient’s address, but the domain either doesn’t exist or isn’t set up to receive mail. For Invalid Recipient bounces, we send the request to the domain, but we get a reply that the recipient doesn’t have an account on that server — i.e., the address does not exist. In both cases, we expect these addresses to remain dormant in the future.

Hard Bounce

The response rates for these two categories were so small you can barely see them on a 100-point scale, so let’s zoom in a bit:

When we shrink our top Y-axis value to 2%, you can see that a very small portion of recipients do eventually engage. The data shows Invalid Recipient bounces are about twice as likely to engage as those with DNS failures, but both have very low engagement rates. Even at one year, Invalid Recipients have just under a 1.7% chance to have opened a message.

Our recommendation: just don’t. Even if you have 1-2% of recipients who may be valid addresses, the damage you do by retrying that 98-99% of addresses will far outweigh any benefits.

This research should provide you with a nice overview of how bounces should impact your decision on re-engaging recipients. It’s important to understand that different types of bounces mean different things, and not to have a blanket policy one way or the other. Let the data direct your future email strategy.

If you liked this analysis, stay tuned! We’ll be expanding on this data in one (or more) future posts where we dive deeper into specifics like mailbox provider and bounce type.