Last updated by Sascha Brockel on 22 December 2022
Expected Reading Time: 18 minutes
Anyone who owns their own domain knows the pain. Emails may end up in spam for the recipient. And that even though the message may not even have an attachment or images. The reasons for this can be manifold. Maybe the domain has been blacklisted. Maybe you are just missing the right configuration.
So what can you do about it? I'll show you in this article with a detailed guide. I speak from experience, as I have struggled with this myself and had to painstakingly search for everything. Since I have also been helped more often by websites that have given a list of possible solutions, I would like to do the same for you.
How can I test if my mails end up in spam?
This can be easily tested by simply sending an e-mail from an e-mail address with your own domain to another e-mail address of yours or that of a friend. So for example from [email protected] to [email protected]. You can simply write Test in the subject line and "This is only a test" in the message. Large providers such as Gmail or Outlook are recommended, as they take rigorous action against spam emails.
If the email ends up in spam, the whole thing often looks like this in the inbox:
If we now take a closer look at the email by opening it, we immediately see a large warning. This hides HTML content and images until we give our approval at our own risk and in this case click on "looks safe".
So now we know that our emails end up in spam. If this already happens with such a standard mail, then we have a basic configuration problem. Many points can be responsible for this. We will now go through all of them one by one. It is worth implementing all the points, as they all increase the credibility on the Internet for spam testing.
The blacklist for e-mails (spam indicator)
A blacklist is an ordinary list of entries. All entries in this list are not allowed and will be excluded. If your domain or IP address ends up on it, your emails will most likely end up in spam.
Is my domain on a blacklist?
This question can be answered quickly with the help of tools from the Internet. Anyone can check this for any available domain on the Internet. Thereby the Email Blacklist Tool from MXToolbox. Simply enter your domain and click on the "Blacklist Check" button. The system will then check your domain and hopefully it should look like the image below.
If blacklist entries have been found, we have to remove them and, first and foremost, make sure that your domain does not end up on the list again. But for this we remove the entries first.
Why is my domain blacklisted?
This can have many possibilities. But here is a list of things that may be responsible for this:
- Spam words in subject or email text such as limited offer
- Exclusively capitalized words such as BUY NOW
- Subject that has nothing to do with the actual email text
- Too many embedded images in the email text (not as attachments)
- Messages from users about received spam mails
- Many exclamation marks e.g. Now available!!!!!
How do I get the domain removed from the blacklist?
If the domain was really blacklisted at any provider, then usually only direct communication helps with the removal of the entry. You should look for the service and a corresponding contact option and ask for the entry to be removed.
Many providers also already provide a form that is intended precisely for this purpose. In this form, you then simply enter the domain, the reason for the removal of the entry and your contact details, often in the form of name and e-mail address, and the entry is removed.
My mails end up in spam without a blacklist entry - what to do?
This problem is well known to me, because I had misconfigured something in my DNS (Domain Name System) records, respectively something was missing. In the simplest case it might be a missing CNAME entry in the DNS records, as it was the case with me among others. So that the e-mails could find their correct way, I needed the following entry.
mail.sascha-brockel.de. 1 IN CNAME ssl0.ovh.net.
Even if it should actually be enough that you had deposited the MX records (Mail Exchange), it was not the case with the provider OVH. So it is also possible that you have similar problems with other providers. In case of doubt, you have to check this configuration with the respective provider.
Check MX records
An MX record forwards e-mails to a mail server. If the correct Mail Exchange entry expected by the recipient is not specified here, this usually leads directly to recognition as spam e-mail. The mails arrive, but do not go through the DNS path intended for them.
This is similar to placing a forwarding order with the post office to ensure that your letters are delivered correctly after you move.Google (2022), https://support.google.com/a/answer/33915
You can find the MX record at the hoster of your domain, e.g. Strato, IONOS, OVH, Google, etc.. If you have already moved your DNS records, then you need to look there. For example, I currently have my domain at OVH, but the DNS records at Cloudflare. The domain hoster specifies the MX records. You can find them out quickly by googling.
There is an exception if you use a service like Cloudflare's email forwarding. Then the MX records must be set to the values required by Cloudflare for the service to work. This may also be required with other providers. Most of the time, however, instructions explain exactly what needs to be done.
If you want to check the correctness of the entries, you can use a tool from Google. And this is Check MX records from the Google Admin Toolbox. There you only have to enter your domain including extension, so in my case sascha-brockel.com. If everything is configured correctly, it should look similar to the image below.
The point "No Google Mail exchangers were found" can be safely ignored, as it is only relevant for people with Google Workspace. In other words, if the domain is also with Google. We will come to the point of the missing DKIM setup later. The same also applies to the successful formatting of the DMARC guidelines here, which you may not have set up yet.
Now the MX records are successfully set up, unless the check has shown something else. Then you can work through the above points again. Now e-mails arrive in any case. But as a detection against spam this is not enough.
Spoofing & Phishing
Spoofing can be divided into domain spoofing, IP spoofing and e-mail spoofing. In line with the topic, however, we will now only deal with e-mail spoofing. In email spoofing, an attacker uses an email header to disguise his own identity and impersonate the legitimate sender. Scripts are used to forge the fields that an email recipient can see.
This means the From e-mail address. Unfortunately, the e-mail transmission protocol Simple Mail Transfer Protocol (SMTP) does not check against such manipulation. SMTP never checks the e-mail header, only the "SMTP envelope". And this is exactly what the recipient does not see. But as a domain owner, you can protect yourself against this with a few DNS entries. These follow after this topic.
Phishing is the sending of fake emails that aim to trick people into falling for a scam. There are also variants where the focus is on stealing confidential information in order to use or sell this information. Thus, the attacker would pretend to be your domain or email address, respectively, and gain trust accordingly, because it seems that you have sent this email.
SPF entry against spoofing
An SPF (Sender Policy Framework) record lists the servers that are authorized to send email from a particular domain. So, if someone invents an email address associated with a domain, it will not be listed in the SPF record and cannot be authenticated.Cloudflare (2022), https://www.cloudflare.com/de-de/learning/dns/dns-records/dns-spf-record/
The Sender Policy Framework, or SPF, lets you specify all the IP addresses and hostnames that are allowed to send email for your domain. If SPF is not configured for the domain, your emails may be blocked or marked as spam by email recipients. Accordingly, this entry is a must-have against spam.
You can create such an SPF record for your DNS records. As already explained for the MX record above, you can find the DNS records at the hoster of your domain or the provider to which you have moved the DNS records. An SPF record looks like this:
v=spf1 include:_spf.mx.cloudflare.net include:mx.ovh.com -all
The entry is then a TXT entry. TXT records can be used to store any text that a domain owner wants to associate with their domain. There is no SPF entry as such. As name you simply enter your domain, so that the entry is valid for the whole domain. The export looks like this:
sascha-brockel.com. 1 IN TXT "v=spf1 include:_spf.mx.cloudflare.net include:mx.ovh.com -all".
Let's solve the formula once.
v=spf1 tells the server that the text contains an SPF record. Then follows the part which says which providers are authorized to send e-mails on behalf of the domain. Say here are
include:_spf.mx.cloudflare.net include:mx.ovh.com. But you can also instead of
include the keyword
ipv4 which then looks like this
ip4=126.96.36.199. This is then tested directly against the specified IP addresses, or in this case only against one, which is allowed to send e-mails on behalf of the domain.
Now we are missing the end with
-all. Here one can distinguish between three variants:
- Addresses that are not listed in the SPF entry are not authorized to send e-mails and should be rejected.
- Unlisted addresses should be marked as unsafe or spam, but still accepted
- Any server/address may send emails on behalf of the domain
There can be only one SPF entry per domain! The entry must be created with the variable
all end or a variable
redirect: contained. This indicates that the SPF record is hosted by another domain.
Why multiple entries at Cloudflare?
As you can easily see, I refer to an address from Cloudflare as well as OVH in the example. The reason for this is that I use Cloudflare's email forwarding service. So Cloudflare needs to be authorized to send email, even if they are still initially coming from OVH. Additionally I also need a CNAME record for the subdomain mail.sascha-brockel.de:
mail.sascha-brockel.de. 1 IN CNAME ssl0.ovh.net.
Otherwise, there were problems sending and receiving emails with OVH. Maybe this is a quirk from the domain owner.
DKIM entry against spam
DomainKeys Identified Mail (DKIM) records use a pair of cryptographic keys for authentication: one public and one private. The public key is stored in the DKIM record and the private key digitally signs the DKIM header. Forged emails from a domain with a DKIM record are not signed with the correct cryptographic keys and therefore cannot be authenticated.Cloudflare (2022), https://www.cloudflare.com/de-de/learning/dns/dns-records/dns-dkim-record/
DKIM consists of two parts: the DKIM record, which is stored as a record in the DNS for the domain, and the DKIM header, which is appended to all e-mails from the domain. DKIM uses digital signature techniques based on public key cryptography. You may know this from SSH connections where you also use a pair of cryptographic keys (private key & public key).
Also for DKIM we have to create a DNS entry respectively TXT entry in the DNS settings. A DKIM record looks like this:
Don't worry, you don't have to make it up yourself. For that there are tools like the DKIM Record Generator from EasyDMARC. Here you just enter your domain name, in my case sascha-brockel.de, a self-selected selector, which you can call as you like, and you can leave the key length at 1024.
But one thing is missing in the entry by default. I now draw attention to this in the keyword breakdown.
v=DKIM1 tells the server that this contains a DKIM entry.
t=s is optional, but recommended and a bit technical. Now the part you should complete manually:
k=rsa is the encryption algorithm, which is always RSA. Last but not least is the public key
The important thing when inserting as a TXT entry is the selector you have chosen. In my case, it is as shown in the picture "test". The name for the entry must therefore be test._domainkey are. DKIM entries always end with _domainkey. Finally, the whole thing looks like this in my exported DNS record:
test._domainkey.sascha-brockel.de. 1 IN TXT "v=DKIM1;t=s;k=rsa;p=MIGfMA0GCSqGSIb3DQEBAQUBB4GNADCBiQKBgQDWd7m0/outbPZPDp10SY8wpLKoHiNLiap6FvO3Ur0qHcMHmqZbKI1McugQNGttAeIPF4dAWiibtYvsZtgJrQMgTBY7gwuMUoSD23b/z9OoSExbi6PIO51/10JaCpvHa4wE/Gs2CBucGsVBSIpmm0EgpwniaQ8nUad5lV6QcZq/wwIDAQAB"
Test if the DKIM entry works
To test if DKIM is really set up and works as expected we can use the tool DKIM Record Checker from EasyDMARC. Here we simply enter our domain and our self-selected selector. We should then receive the output of our TXT entry. If the message "no selectors detected" appears instead, something is wrong. It may take a moment for the domain host to publish the TXT entry. You can check this with the Supertool of MXToolbox.
Now also the check at Google would have Check MX records shows that DKIM is successfully set up if you have specified the DKIM selector.
The private key must then be installed on your mail server and/or your domain hoster for the e-mails. Without this, the whole thing does not work. The installation differs with all providers, but can be quickly googled.
DMARC entry in combination with SPF & DKIM against spoofing
DMARC (Domain-based Message Authentication Reporting and Conformance) records contain DMARC policies that tell email servers what to do after checking SPF and DKIM records. Domain owners can set rules for whether to block, allow or deliver messages based on these checks. Because DMARC policies are checked against other authentication policies and allow domain owners to set more specific rules, these records provide additional protection against email spoofing.Cloudflare (2022), https://www.cloudflare.com/learning/dns/dns-records/dns-dmarc-record/
The DMARC policy determines whether the email is marked as spam, blocked or delivered to the intended recipient. First, SPF & DKIM are tested using the header and DMARC now lets us decide how we want to proceed if the tests pass and if not. A DMARC entry looks like this:
v=DMARC1; p=quarantine; fo=1; rua=mailto:[email protected];
What does that mean now?
v=DMARC1 indicates that this TXT entry contains a DMARC policy.
p=quarantine specifies that emails that do not pass DKIM and SPF should be quarantined and considered spam. Other possible settings for this are
p=none, which sends emails that did not pass anyway.
p=reject in turn, instructs email servers to block emails that have not passed.
fo=1 generates a DMARC error report if SPF and/or DKIM fail the test. If you now want to receive the error reports to a specific email address, you have to
rua=mailto:[email protected] complete.
The settings for DMARC can be set much more fine-grained. That is, percentages for filtering, test modes for SPF and DKIM and more. The so-called DMARC Tags you can see here and apply according to your needs.
Again you have to create a TXT entry. The name for the entry must start with _dmarc start. This is followed by your domain in the regular way. So the entry is perceived by DMARC check mechanisms. This looks then as DNS entry export with me in such a way:
_dmarc.sascha-brockel.de. 1 IN TXT "v=DMARC1; p=quarantine; fo=1; rua=mailto:[email protected]"
BIMI entry as a nice-to-have
A Brand Indicators for Message Identification (BIMI) entry is another DNS entry pushed by large manufacturers. From your smartphone, you surely know e-mails in which you see a circle on the recipient's screen, where the sender's photo may be stored if you have saved it or, for example, the Google logo if you receive an e-mail from Google. You can do exactly the same for your branding with your logo.
But this is quite complex, because the logo must meet quite a few requirements to be used then. It must be a SVG file (vector graphic) in Tiny format and much more. I can write a separate article about this if necessary. Otherwise it would go beyond the scope. BIMI works only in cooperation with DMARC. Accordingly, the logical path of the configuration is to set up SPF followed by DKIM followed by DMARC and last but not least then BIMI.
To check if you have configured everything correctly and your entries fit, there is an all-in-one tool from MXToolbox called Email Deliverability. The whole thing is very easy to use. You send an e-mail from your e-mail address or domain to be tested to [email protected] and you will receive an email from MXToolbox with your results.
There you can then go to a page where you will get complete results about whether everything worked. You can see all email headers and even the time taken by each endpoint and if they are blacklisted. For me, at the time of the screenshot, DKIM was not set up on the mail server and therefore failed.
Example implementation in YouTube video
The video will follow. Here we will apply the whole thing once practically on the basis of an example.
Long story short, no point... If you have gone through these steps and set everything up, you are well protected against getting your domain into blacklists and therefore not into the spam folder. These DNS entries are elementary for the credibility of your domain. Especially since there are more and more spammers and scammers, it is very important.
I hope I could help you further. Feel free to visit my YouTube channel. If you have any questions or comments, feel free to post a comment.