How to Build a Site Google and AI Can Actually Read
You can write the best content in your industry and still get no traffic from it. I have watched it happen dozens of times. The pages are helpful, the research is solid, the writing is clear, and yet Google barely sends a visitor. Almost every time, the problem is not the content. It is the plumbing underneath it.
That plumbing is technical SEO. It is the part of search that decides whether a search engine can even reach your pages, understand them, store them, and trust them enough to show them. If that layer is broken, nothing above it matters. And in 2026 there is a new reason to care about it, because the same machinery that lets Google index your site is now what lets ChatGPT, Perplexity, and Google’s AI answers find you and cite you.
This is the checklist my team at Admonq runs on every site we take on. I will walk you through all of it: what technical SEO is, how search engines process a page, the full checklist grouped the way I work through it, the tools I use, how to run an audit, and the mistakes I see most often. By the end you will look at any site, including your own, and know exactly what to fix first.
Technical SEO is the foundation every other layer sits on. Break a lower layer and everything above it falls.
What is technical SEO, exactly?
Technical SEO is the work of making your website easy for search engines to crawl, render, index, and trust. It has nothing to do with what your content says and everything to do with whether a machine can access and process it without tripping over something.
It helps to split SEO into three buckets. On-page SEO is about the content on a page: the words, the headings, the images, how well it answers a query. Off-page SEO is about signals from the rest of the web: links, brand mentions, reputation. Technical SEO is everything else, the site-wide foundation that lets the other two work at all. Fast loading, clean code, a logical structure, secure delivery, no broken links, no accidental blocks. It is the least glamorous of the three and the one that quietly decides how far the other two can go.
Here is how I explain it to clients. On-page and off-page SEO are how you compete once you are in the race. Technical SEO is what gets you to the starting line. If a search engine cannot crawl a page, your keywords do not matter, and if it loads in eight seconds on a phone, your effort leaks out through a slow experience. Technical SEO removes the reasons a search engine has to ignore you.
The good news: most of it is a solved problem. Unlike content, which needs constant judgment, technical SEO is largely a checklist. You work through it, fix what is broken, keep it maintained, and move on. That is why I built this guide as a checklist rather than a lecture.
Why does technical SEO matter more in 2026 than it did five years ago?
For years, technical SEO was something you set up once and mostly forgot. That has changed, for three reasons.
The first is mobile. Google indexes and ranks the mobile version of your site, not the desktop one. This is not new, but plenty of sites still treat mobile as an afterthought. More than sixty percent of searches now happen on a phone, and Google grades your speed on a mid-range Android device on a normal connection, not the fast laptop your developer tests with. If your mobile experience is weak, that is the experience Google judges you on.
The second is speed as a measured signal. Google no longer takes your word for how fast your site feels. It measures the experience of your real visitors through Core Web Vitals, and it uses those numbers as a ranking input. I will cover the exact thresholds later, but the headline is simple: performance is now a number Google can see, not a vibe you can claim.
The third reason changed everything, and it is why I keep telling clients that technical SEO and AI visibility are now the same project. AI search tools do not invent knowledge about your business. They read the web with crawlers, the same way Google does, and they can only cite pages they were able to reach and understand. If your site blocks their crawlers, hides content behind heavy scripts, or buries pages where nothing links to them, you are invisible to AI answers just as surely as you are to Google. This is the bridge to the newer discipline of generative engine optimization. GEO is how you earn citations in AI answers, but it only works if the technical foundation lets the machines in. A crawlable, fast, well structured site is the price of admission for both.
The short version Technical SEO used to be about pleasing one crawler, Googlebot. In 2026 you are building a site that a whole set of machines can read: Googlebot, plus GPTBot, Google-Extended, PerplexityBot, ClaudeBot, and more. Get the foundation right and you serve all of them at once.
How do search engines crawl, render, and index a page?
Before the checklist makes sense, you need the mental model. Every page that shows up in search, or gets cited by an AI tool, passes through four stages. When a page is not showing up, it failed at one of them. Diagnosing technical SEO is mostly finding which stage broke.
Every page runs this gauntlet. A failure at any gate means the page never reaches search or AI answers.
Crawling is discovery. A bot follows links and reads your sitemap to find URLs, then requests them from your server. A page gets crawled only if the bot can find it and is allowed to fetch it. Pages break here when they are blocked in robots.txt, when no internal link points to them, or when the server returns errors.
Rendering is construction. Modern pages are rarely just plain HTML. The browser, and the crawler, has to run the page’s code, load its styles and scripts, and build the final version a user would see. This matters enormously for sites built on JavaScript frameworks. If the important content only appears after a pile of script runs, the crawler has to do extra work to see it, and sometimes it does not wait around. Pages break here when they lean too heavily on client side rendering, when scripts or stylesheets are blocked, or when the page is simply too slow to build.
Indexing is storage. Once a page is crawled and rendered, the search engine decides whether to keep it in its index, the giant library it searches when someone types a query. Not every crawled page gets indexed. Pages break here when they carry a noindex tag by accident, when a canonical tag points somewhere else, or when the content is thin or nearly identical to other pages.
Ranking and citation is competition. Only now, once a page is safely in the index, does it compete for positions and for citations in AI answers. This is where content quality, relevance, links, and Core Web Vitals do their work. But notice how much has to go right before you reach this stage. Most of what people call an SEO problem is actually a crawl, render, or index problem in a costume.
Keep this four stage model in your head as you read the checklist. Every item below is really about clearing one of these gates.
The technical SEO checklist for 2026
This is the core of the guide. I have grouped the checklist the way I actually work through an audit, from the foundation upward. Do not skip ahead to the shiny performance items. If Google cannot crawl or index a page, making it faster is polishing something nobody will ever see. Fix the base first.
The full checklist at a glance. Work top to bottom, left to right. Crawl and index issues come first.
Crawlability: can bots reach your pages?
Start with your robots.txt file at yoursite.com/robots.txt. This tiny text file tells crawlers where they may and may not go. It is astonishing how often I find a site accidentally blocking its own important sections here, usually a leftover rule from a staging environment that shipped to production. Open it, read every line, and make sure you are not disallowing anything you want found. While you are there, confirm you are not blocking the AI crawlers you want citing you, more on that shortly.
Next, your XML sitemap. This is a file that lists the URLs you want indexed, and it gives crawlers a clean map of your site instead of making them find everything by following links. Generate one, keep it current, and submit it in Google Search Console. A good sitemap contains only canonical, indexable URLs that return a normal success response. If your sitemap is full of redirects, error pages, or blocked URLs, you are sending mixed signals.
Then check for crawl errors directly. Google Search Console has a report that shows you exactly which pages Google tried to crawl and could not, and why. This is the single most useful free diagnostic in SEO. Work through those errors: fix broken links, repair redirect chains, and clean up pages returning server errors. On larger sites, think about crawl budget, the amount of crawling Google will spend on your site. You protect it by not wasting crawls on junk URLs, endless filter combinations, and duplicate pages.
Indexation: are the right pages getting stored?
Crawling gets a bot to your page. Indexing decides whether it is kept. The most common indexation bug I find is the accidental noindex tag, a small instruction in the page’s code that tells search engines to leave it out of the index. It has a place, you want it on thank you pages and internal search results, but I regularly find it left on pages that desperately want to rank. Check your key templates and make sure none of them are quietly telling Google to ignore them.
Canonical tags are the next piece. A canonical tag tells search engines which version of a page is the real one when several similar URLs exist. Used well, it consolidates duplicate pages into one strong signal. Used carelessly, it points your important pages at the wrong version and tells Google to index something else. Audit them so every page either points to itself or deliberately points to a chosen master.
Finally, prune. Thin pages, near duplicate pages, and low value pages that nobody searches for do not just fail to rank. In bulk they dilute the overall quality signal of your site and waste crawl budget. I would rather have two hundred strong pages than two thousand weak ones. Find the pages that get no traffic and no links, and decide deliberately whether to improve them, merge them, or remove them.
Site architecture: is your structure logical and shallow?
Site architecture is how your pages connect to each other. Get it right and both users and crawlers can find anything quickly, and link authority flows smoothly through the site. Get it wrong and your best pages sit buried where almost nothing points to them.
Flat wins. The deeper a page sits, the less often it gets crawled and the weaker its signals.
The rule I live by is simple: no important page should be more than three clicks from the homepage. This is a flat architecture, and it works because link authority passes through internal links. Your homepage is usually your strongest page. Every click away from it, a little of that strength gets diluted. A page buried five or six clicks deep receives almost none of it, and crawlers visit it rarely, if at all.
Internal linking is the tool that makes this happen. When you publish something new, link to it from relevant existing pages, and link out from it to related content. This does two jobs at once. It passes authority to the new page, and it helps search engines understand how your content relates, which topics you cover in depth, and which pages are the important hubs. This is also where your content strategy and your technical structure meet. The topic clusters you plan during keyword research should map directly onto your internal linking, with a strong pillar page linking out to the supporting posts around it.
URLs matter here too. Keep them short, readable, and descriptive. A URL like yoursite.com/technical-seo-checklist tells a human and a machine what the page is about; yoursite.com/p?id=48213 tells them nothing. Use words with hyphens between them, avoid unnecessary parameters, and once a URL is earning links, do not change it without a proper redirect.
Page speed and Core Web Vitals: is your site fast on real devices?
Speed is where technical SEO becomes measurable, and where a lot of sites lose ground without realizing it. Google measures three specific things, the Core Web Vitals, using data from real Chrome users rather than a lab test. That distinction matters: a perfect score in your developer tools means nothing if a quarter of your actual visitors, on ordinary phones, get a slow experience. Google grades you at the seventy fifth percentile of real visits, so you have to be good for most people, not just the lucky ones on fast connections.
The three Core Web Vitals and their 2026 thresholds. You need all three in the green to pass.
There are three metrics, and the thresholds have not changed for 2026. Largest Contentful Paint, or LCP, measures loading. It is the time until the biggest visible thing on the page, usually a hero image or a headline block, finishes loading. Good is 2.5 seconds or less. You improve it by serving a fast main image, using modern formats like WebP or AVIF, cutting server response time, and preloading the important resources.
Interaction to Next Paint, or INP, measures responsiveness. It captures how quickly the page reacts when a user taps or clicks, across the whole visit. Good is 200 milliseconds or less. INP replaced the older First Input Delay metric in 2024, and it is the metric most sites now fail, because fixing it usually means rethinking JavaScript rather than just compressing an image. You improve it by shipping less script, breaking up long tasks so the main thread stays free, and deferring anything that does not need to run immediately.
Cumulative Layout Shift, or CLS, measures visual stability. It scores how much the page jumps around as it loads. You have felt bad CLS every time you went to tap a button and an ad loaded and pushed it away. Good is 0.1 or less. You fix it by setting explicit width and height on every image, video, ad slot, and embed, and by reserving space for anything that loads in late so it does not shove the rest of the page down.
Where to start If you are failing more than one Core Web Vital, fix whichever is in the red band first, then tackle INP, then LCP, then CLS. And remember the data updates on a rolling window of about a month, so give a fix several weeks before you judge whether it worked.
Mobile and rendering: does your content survive on a phone and through JavaScript?
Since Google indexes the mobile version of your site, your mobile experience is not a nice to have, it is the version that gets ranked. Make sure your design is fully responsive, that text is readable without zooming, that tap targets are not crammed together, and, critically, that your mobile version contains the same content as your desktop version. I have seen sites hide whole sections on mobile to look cleaner, not realizing they were hiding that content from the index too.
Rendering is the subtler issue, and it trips up modern sites built on frameworks like React, Vue, or similar. If your content is generated in the browser by JavaScript, a crawler has to run that code to see it. Google can do this, but it is slower and less reliable than reading plain HTML, and other crawlers, including some AI bots, are far less capable at it. The safest approach is to serve your important content in the initial HTML, through server side rendering or static generation, so it is visible the moment the page is fetched, no script execution required. A quick test: load a page with scripts disabled. If your main content vanishes, it vanishes for the machines that render poorly too.
Structured data: does your markup tell machines what your content means?
Structured data, also called schema markup, is code you add to a page that spells out what the content is in a language search engines understand. It says, in effect, this is a product with this price, this is an article by this author published on this date, this is a recipe, this is a set of frequently asked questions. Search engines already try to work this out from your text, but structured data removes the guesswork.
The immediate payoff is rich results, the enhanced Google listings with star ratings, prices, and FAQs that stand out and earn more clicks. The 2026 payoff is bigger: clear, machine readable markup also helps AI systems parse your content accurately and cite it correctly rather than misquote it. It is one of the cleanest overlaps between traditional SEO and AI visibility. I go deep on implementation in my schema markup guide, so I will keep this short: at minimum, add Article schema to your posts, Organization schema to your site, and FAQ schema wherever you answer common questions.
Security and trust: is every page served over HTTPS?
HTTPS, the secure version of the web protocol shown by the padlock in the browser, has been a ranking signal for years and today is simply expected. Every page should load over HTTPS, with no mixed content, which is when a secure page loads an insecure image or script and browsers flag it. If you are still on HTTP anywhere, moving to HTTPS is a foundational fix, not an optional one.
Trust runs deeper than the padlock. Search engines increasingly weigh who is behind the content and whether it is credible, the qualities Google groups under experience, expertise, authoritativeness, and trust. Clear authorship, an about page, accurate contact details, and a real track record all feed this. It sits at the border of technical and content SEO; I cover the content side in my piece on building E-E-A-T into your content. On the technical side, make sure your authorship markup, organization details, and secure delivery all line up.
AI crawler readiness: can the answer engines reach you?
This is the newest section of any technical SEO checklist, and in a year it has gone from optional to essential. The AI tools that people increasingly search with, ChatGPT, Perplexity, Google’s AI answers, and others, use their own crawlers to read the web. If you want to be cited in those answers, you have to let those crawlers in and make your content easy for them to parse.
Start with robots.txt again, this time from the opposite angle. Check whether you are allowing the AI crawlers you care about. The main ones to know are GPTBot from OpenAI, Google-Extended which governs Google’s AI training and answer use, PerplexityBot, ClaudeBot and anthropic-ai from Anthropic, and CCBot from Common Crawl, which feeds many AI systems. Blocking them is a legitimate choice if you do not want your content used for AI, but understand the trade off: a blocked crawler cannot cite you. If AI visibility is a goal, allow the ones that matter to you.
There is also an emerging standard called llms.txt, a file at your site root that offers AI systems a clean, structured guide to your most important content, much like a sitemap does for search crawlers. Adoption is still early and no one should promise guaranteed results yet, but it is cheap to implement and signals that you are thinking ahead. Everything else in this checklist already serves the AI crawlers too, because clean HTML, fast pages, logical structure, and clear markup help every machine that reads your site. This is the practical heart of generative engine optimization: the technical foundation is what makes AI citation possible in the first place.
Which technical SEO tools do I actually use?
You do not need an expensive stack. A handful of tools, most with a free tier, cover the vast majority of technical SEO work. Here is what I reach for and what each is good at.
| Tool | What I use it for | Cost |
|---|---|---|
| Google Search Console | Crawl errors, index coverage, Core Web Vitals on real data, which queries you show up for. The single most important tool, and it is free. | Free |
| PageSpeed Insights | Checking Core Web Vitals for a specific page and getting concrete fix suggestions, using both field and lab data. | Free |
| Screaming Frog | Crawling your whole site the way a search engine would, to surface broken links, redirect chains, missing tags, duplicate content, and noindex issues at scale. | Free tier, then paid |
| Ahrefs or Semrush | Full site audits with prioritized issue lists, plus the backlink and keyword data that connects technical work to the rest of your SEO. | Paid |
| Rich Results Test | Confirming your structured data is valid and eligible for rich results before you rely on it. | Free |
My honest advice is to start with the free ones. Search Console, PageSpeed Insights, and a crawl from the free tier of Screaming Frog will surface almost every technical problem on a small to mid sized site. Reach for paid platforms when you need scale, monitoring, or the wider keyword and backlink picture. The category is shifting fast as AI features arrive in these tools, which I cover in my roundup of the best AI SEO tools.
How do you run a technical SEO audit from scratch?
Here is the process I follow on a new site, in order. It mirrors the crawl, render, index, rank model, and it stops you from wasting time polishing things that do not matter yet.
- Crawl the whole site. Run Screaming Frog or a similar crawler across every URL. This gives you the raw inventory: every page, its status, its tags, its links. Almost every problem shows up somewhere in this crawl.
- Check indexation. In Search Console, look at how many pages are indexed versus how many you expected, and read the reasons pages were excluded. Big gaps here point straight at crawl or index problems.
- Review crawlability. Read robots.txt line by line, confirm your sitemap is clean and submitted, and work through the crawl error report.
- Test Core Web Vitals. Pull the Core Web Vitals report in Search Console for the real picture, then run your key page templates through PageSpeed Insights for specific fixes.
- Audit architecture and internal links. Look at how deep your important pages sit and whether they are well linked. Find orphan pages that nothing points to.
- Validate structured data and HTTPS. Confirm your markup is valid and your whole site is served securely with no mixed content.
- Check AI crawler access. Confirm your robots.txt allows the AI crawlers you want, and consider adding an llms.txt file.
Once you have your list of issues, the question becomes what to fix first, and this is where most audits stall. You end up with fifty problems and no idea where to begin. I prioritize everything on two axes: how much impact the fix will have, and how much effort it takes. Plot each issue on that grid and the order becomes obvious.
Plot every issue by impact and effort. Start in the top left. Quick wins first, big projects planned, low return work last.
The top left quadrant, high impact and low effort, is where you start. Fixing an accidental noindex on an important template, submitting a missing sitemap, or repairing broken internal links can take an afternoon and move real numbers. The top right, high impact but high effort, is your project backlog: improving Core Web Vitals, restructuring a messy architecture, or fixing a rendering problem, all worth doing but needing planning. The bottom left are small tidy ups for spare time. And the bottom right, low impact and high effort, is where good intentions go to die. Chasing a perfect hundred on a lab score, or endlessly tuning pages nobody searches for, feels productive and rarely is. Spend your time where impact is high.
The technical SEO mistakes I see most often
After enough audits, the same mistakes show up again and again. Watch for these.
Blocking your own site by accident. A noindex tag or a robots.txt rule left over from development, shipped to the live site, quietly keeping important pages out of Google. It is the most common serious bug I find, and the most damaging, because everything looks fine until you notice the traffic that never arrived.
Chasing lab scores instead of real ones. Obsessing over a perfect score in a testing tool while your real visitors, measured in the field, still have a slow experience. Google grades the real data. Optimize for that.
Ignoring mobile. Testing everything on a desktop and assuming mobile is fine. Google ranks the mobile version, so that is the one that needs to be fast, complete, and usable.
Letting thin pages pile up. Publishing endlessly without ever pruning, until thousands of weak pages drag down the quality signal of the whole site and soak up crawl budget. Quality of the overall index beats raw quantity.
Breaking URLs without redirects. Changing URLs during a redesign without setting up redirects, throwing away every link and bit of ranking those pages earned. If a URL must change, redirect the old one to the new one.
Treating technical SEO as a one time job. Fixing everything once and never checking again. Sites drift. New content, plugin updates, redesigns, and third party scripts all introduce new issues. Technical SEO is maintenance, not a single event.
The one to check today If you do nothing else after reading this, open Search Console and look at your index coverage, then read your robots.txt file. Accidental blocks are common, quietly devastating, and usually fixable in minutes. It is the highest return five minutes in SEO.
Technical SEO FAQ
How long does technical SEO take to show results?
It depends on what you fix. Removing an accidental block that kept pages out of the index can show results within days, as soon as Google recrawls. Core Web Vitals improvements take longer, since the data updates on a rolling window of about a month. Architecture changes work gradually as the site is recrawled. In general, technical fixes show up faster than content or link building, because you are removing a barrier rather than building new authority.
Do I need to know how to code to do technical SEO?
For diagnosis, no. The tools in this guide surface most problems without you writing a line of code. Some fixes, especially rendering and Core Web Vitals work, need development skills or a clear brief to a developer. But much of technical SEO is checking settings, editing text files like robots.txt, and configuring your content management system, all well within reach for a non developer.
Is technical SEO different for AI search than for Google?
The foundation is the same, which is the good news. A crawlable, fast, well structured site with clean markup serves Google and the AI crawlers alike. The main additions for AI are making sure you allow the AI crawlers in your robots.txt, considering an llms.txt file, and being extra careful that your content is in the HTML rather than hidden behind scripts, since some AI crawlers render JavaScript poorly. Get the classic technical foundation right and you are most of the way there for AI too. The rest is covered under generative engine optimization.
What is the single most important technical SEO factor?
Crawlability and indexation, without question. Every other factor assumes your pages can be reached and stored. A blazing fast page with perfect markup is worthless if a stray noindex tag keeps it out of the index. Always make sure search engines can actually crawl and index your important pages before you optimize anything else.
How often should I run a technical SEO audit?
For most sites, a full audit twice a year plus a quick monthly check of Search Console for new crawl and index errors is a sensible rhythm. Run an extra audit any time you make a big change: a redesign, a platform migration, a large batch of new content, or a URL restructure. Those are exactly the moments technical problems get introduced.
Want this done for you?
At Admonq we run this exact checklist on client sites, fix what is holding them back, and build the technical foundation that ranks in Google and earns citations in AI answers. If you would rather have experts handle it, let us take a look at your site. Get a technical SEO audit
Technical SEO is not the most exciting part of digital marketing, but it is the part that decides whether everything else works. Get the foundation right, keep it maintained, and every piece of content you publish and every link you earn will have a clear path to do its job, in search results and in the AI answers that are becoming the new front page of the web.
