You received a confusing and potentially very concerning email from Google about your website. This part sounds especially worrisome:
Google Search Console has identified that some pages on your site are not being indexed due to the following new reason: Not Found 404.
What??? Why is Google not indexing your pages? What is preventing them from being found? What happened to your website? All questions I would have if I took Google at their word. Before you let Google’s robots stress you out, though, it’d be better to fact check. Because sometimes they lie.
None of this 404 stuff is rocket science, but the wording in that Search Console email is enough to trip anybody up. So while there are a number of things you’ll need to know to get (and stay) on top of the situation, let’s start with the high-level basics:
- Not all 404 errors are problematic, and may not require immediate action if they correspond to pages that aren’t meant to be indexed.
- It’s essential to evaluate whether the 404 errors Search Console is reporting have any significant implications for search engine optimization or overall user experience. Sometimes the URLs literally never existed.
- Checking the URLs Search Console identified will help you spot any patterns or common causes behind the errors, e.g., broken links pointing to non-existent pages.
- It’s important to address errors – especially those that aren’t intentionally meant to end with a 404. This could involve redirecting or fixing broken links, creating custom 404 error pages, or finding alternative content to replace bad external links.
My aim with this article is to encourage and equip you to confidently deal with (or dismiss, in some cases) the bad URLs Search Console reports. I’ll explain the different types of problems that lead to 404 errors, suggest ways to deal with each, and walk you through ways to keep an eye out for 404s. Regularly monitoring and addressing broken links will keep visitors happy, which will make Google happy and less likely to send you anxiety-inducing e-mails.
Sound like a plan? Then let’s get to it.
What is a 404 error?
I mean, beyond the part of it we all know and hate – the (usually) ugly page you see instead of the one you expected when you clicked a link.
A ‘404 error’ occurs when the browser can’t find something it’s been asked to fetch. That can be distressing for a website owner and annoying for visitors. Instead of finding content you expect, you often get something like this:
Sometimes a mistake or oversight is to blame. But it’s not always your mistake, nor is it always something you can completely control or fix.
Understanding the Main Causes of 404 Errors
A 404 error means that the page or URL – as entered into the browser – doesn’t exist. This can happen for various reasons, such as:
- Broken Links: These are URLs that point to a page that no longer exists, may have never existed, or that has been moved without proper redirection. The visitor ends up in a dead-end – a.k.a., a 404 error. 404s can originate with internal, outbound or inbound links, or images videos and other embedded content.
- Typographic Errors: Visitors mistyping or omitting one character is enough to cause a 404. Or, the same thing happening because you make a mistake while creating an internal link, or another site linked to you (or tried).
- Deleted Content: If you removed a page or a post from your website without setting up a redirect, visitors trying to access that URL will get a 404 error. (We’ll talk about redirects further down the page.) Or, if content you linked to has been moved or deleted, that’ll trigger a 404 as well.
If you think for a minute about the various ways a 404 error could occur, you’ll quickly realize that the vast majority of them originate somewhere other than within the confines of your site. Although that does mean you won’t always be able to directly fix a 404 error, you can take steps to help your visitors find the content they’re looking for.
Types of 404 Errors
Internal 404 Errors
These are bad links within a website. They happen for various reasons, like mistyping a URL, linking to a page you later move, rename or delete, or intending to link out to another site, but mistyping the URL.
Inbound 404 Errors
This happens when someone links to your site using a bad URL. Like internal 404 errors, they can be caused by typos, or because you moved, renamed or deleted the page the other site linked to.
Outbound 404 Errors
Outbound 404 errors are those that originate on your site, caused by all the same reasons as inbound 404s. Maybe you made a typo, maybe the other site deleted the page, or perhaps they just moved it.
Image/Embed 404 Errors
Embedding an image or video that’s later deleted or made private is the usual culprit. Like links in text, URLs for images, videos or other embed code must reach a resource that’s live, or else there’ll be a 404 error.
When to Ignore a 404 Error
- When it’s an isolated case. One person fat-fingered a URL or perhaps just tried to see if you had something on their website they hoped for, e.g., yourdomain.com/free-money.
Yep, that’s pretty much it. One bullet point. It would easy to drive yourself crazy trying to implement a plan for every specific 404 error. Your time is better spent addressing the few 404s you can do something about. Don’t worry about specific fixes for the rest (we’ll cover a general fix further down).
Why you Should Not Use a Broken Link Checker Plugin
Broken link checker plugins for WordPress can have a significant impact on your website performance. These plugins continuously scan your website for broken links, using valuable server resources and putting additional strain on your site’s performance.
The constant scanning and monitoring process can slow down your site’s loading speed, resulting in a poor user experience and potential loss of visitors. In addition, broken link checker plugins tend to generate a large number of WordPress database queries, placing an extra burden on your server and potentially causing timeouts or crashes.
But: If you can stay on top of broken links, you can often fix them before they’re indexed by Google. So, potentially, no more anxiety-inducing GSC emails telling you about missing pages that can’t be indexed. So let’s talk about resources that will help you be proactive instead of reactive about 404 errors –without slowing your site.
Alternatives to Broken Link Checker Plugins
Better options for finding broken links on your website are those that help you perform a one-time check. Yes, that means you’ll need to repeat it from time to time. But as with many maintenance tasks, doing them regularly is quicker and easier than waiting until there’s a big problem.
You can find websites offering “free” broken link checks, but many of them aren’t actually free. Or, they force you to sign up for something before they’ll show you results or let you scan your site. Some may be retaining or displaying your information in ways that aren’t private. So read the fine print before entering your URL on an unfamiliar site. Broken Link Checker is a free option that didn’t require a signup as of the time this article was published.
SEOPress is an excellent WordPress companion for numerous reasons (it really deserves its own blog post), and its broken link check audit is a great value add if you have it (or are in the market for a solid, time-saving SEO tool). The audit will only run when you initiate a scan. So while it is a plugin and does look for broken links, it doesn’t perform constant scans like broken link check plugins do.
SEOPress can optionally run more powerful scans that will tax your server while they’re in progress. Those options are clearly marked with language warning that they will slow your site. If you need them, use them. Just don’t do it unintentionally, or during times of peak traffic.
How to Use Google Analytics (GA4) to Find Broken Links
Unlike Search Console, Google Analytics offers insights that can help track and fix 404 errors. You just need to know where to look and how to interpret the data. Before we get started, remember: We’re not going to worry about one-off 404 errors caused by typos or visitors (or bots) poking around in your site using various non-existing URLs.
Let’s go through the steps the easy way – with pictures 🙂
- Verify the title of your 404 page. This will usually be Page Not Found. Sometimes it’s 404. The best way to know is to intentionally use a URL you know doesn’t exist on your site. Then look in the browser tab for the page title.
- In Google Analytics, go to Reports > Engagement > Pages and Screen. Then, look under the Search field for the option to change the report dimension. It’s not labeled as such, but you’ll see the default dimension with a down arrow to its right. Click it, and choose the “Page Title and Screen Class” option.
- Click the ‘+’ symbol to add Page Path and Screen Class as secondary dimension. Find the ‘+’ next to the “Page Title and Screen Class” dimension you just set. Click it to get a popup with options like those shown in the image below. Click Page / Screen, then choose “Page path and screen class.”
- Search for instances where your 404 page was served. First, change the date range to a time frame long enough to show persistent, recurring 404 errors that should be fixed. At least 90 days. Then, in the search field above the dimensions you set, enter the title of your 404 page from step one. Next, be sure the report is sorting by Views, in descending order.
- Evaluate the URLs under Page path. Start at the top with the URLs with the highest number of attempted Views. Disregard anything with only one view. If that leaves you with more URLs than you think you can deal with in one sitting, export the report to CSV. Then import it to a spreadsheet app so you can track your work.
How to Address 404 Errors
When a 404 error happens because of something on your site, it’s quicker and easier to fix than if it’s coming from another site. But, as I said a moment ago above, most 404 errors don’t directly result from your content. Fixing them will require multiple tactics. Fortunately, none are all that difficult.
Before you begin, you’ll need to look at 404s that Google sees. You’ll have what Search Console emailed you about, but the steps to see 404s in Analytics will tell you where you should actually spend your time. Additionally, you’ll want to scan your website for bad links Google may not have picked up on yet. Don’t skip this last step or it’ll result in more attempted indexing of bad links.
- Correct Internal Links: Correct links that lead to wrong or nonexistent URLs. Think about what content the visitor was expecting, and what would satisfy the need for the content they originally wanted. Sometimes this might mean removing a link altogether.
- Correct External Links: Other sites move or delete pages regularly. You may have to do a little detective work to find a good replacement URL.
- Redirect Deleted Pages: Guide your website visitors and search engines by setting up permanent redirects (301) for any pages you’ve removed or deleted. This sounds ridiculously nerdy if you have never done it, so reach out for help if you need it (it would be a quick, inexpensive task). Using a plugin like Redirection works well enough.
- Check External Backlinks: Reach out to other websites linking to yours, and politely ask them to update any outdated or incorrect links for better accuracy. While you’re waiting on that fix, it’d be best to set up a redirect to send visitors from the bad URL to the correct one.
- Create Custom 404 Error Pages: Design a friendly custom 404 page that helps lost visitors find their way and keeps them engaged on your site.
- Monitor Mistyped URLs: In Google Analytics, keep an eye out commonly mistyped URLs. If it happens more than once, set up a redirect to the correct page.
- Conduct Regular Site Audits: Schedule routine scans with SEOPress or another URL checker, to catch and fix 404 errors before Google or your visitors find them.
Optional (But Recommended): Custom 404 Error Pages
It’s true that you shouldn’t waste much time worrying about one-off errors. Like the person who types ‘bloh’ instead of ‘blog,’ or who tries all kinds of crazy things because they’re really, really hoping to find a free download, read a buried blog post, or any number of things a desperate website visitor might do. Even a sleepy, low-volume site like ours has easily dozens of misdirected visitors each year.
It wouldn’t make sense to spend time addressing specific mixups that aren’t likely to happen ever again, or only once in a blue moon. But you should consider what would help your visitors more than an ugly “Page Not Found” window, or the default WordPress “Oops! That page can’t be found.”
Although a custom 404 page could be an opportunity to have fun, or create something that riffs on your company’s culture, you can also just be super practical. Like, including a search form to help lost visitors find their way. Or linking to your most popular content. Get it done ASAP by enlisting the help of an imaginative pro designer. Brainstorm with them and come up with something perfect for your website and visitors. Or, keep it super simple and DIY with the help of a 404 page plugin.
404 Errors: The Most Important Takeaways
- The fact that Google Search Console sent you an email saying they couldn’t index pages on your site does not necessarily mean there are actual pages that somehow went missing.
- Checking Google Analytics to find out what bad URLs are actually being used, and how often, should be your first step. And, regularly scan your website to find broken links before visitors do.
- Fixing your mistakes, catching others’ typos with redirects, and setting up a great 404 page will keep you, your visitors and Google happy.