How often can my website be crawled?

Your website can be crawled almost as often as you want. The most common intervals are monthly, weekly and daily. The limiting factor is the speed of your website and the number of pages, images, other content and API's used.

Can JavaScript content be rendered?

Yes! The crawler renders JavaScript by default.

What data can be queried?

With our crawler, almost all relevant SEO criteria can be queried and checked. During each crawl of your page, 150 to 200 properties per URL are checked, depending on the configuration and released data source. That is quite a lot! In addition, depending on the configuration and customer requirements, there are dedicated reports on technically more complex issues such as the validation of hreflang. We can include numerous data sources in the crawl in addition to the crawl data of your own website. Provided you provide us with the appropriate login data and API keys. We can include the following data: Google Analytics (Universal Analytics - access to your data required) Google Search Console (access to your data required) Pagespeed Insights (No access or API key required on your part) Majestic (API key required on your part) Ahrefs (API key required on your part) Moz (API key required on your part)

How does the crawler work and where does my data end up?

Our crawler is a Screaming Frog, which is operated in the cloud. The crawls are started automatically on a monthly, weekly or daily basis, depending on the settings. The crawling results are automatically written to Google Big Query and then queried or displayed via Google Data Studio.

Is there an alert or notification function when new problems appear on my website?

No, we do not offer this feature. The crawler is intended for continuous optimization of your website and continuous competition monitoring.

Can a crawl also be triggered manually, for example after a release?

Yes! We can also start the crawler manually on demand.

DIWABE SEO consulting: monitoring, crawling, analysis, sparring and relaunching

Content analysis

Your website content is the critical success factor for successful SEO and the success of your website in general. Technique and structure must fit in order not to inhibit the rankings. Ultimately, however, your content determines whether you actually offer one of the most relevant pages for users for the search queries you are targeting.

We analyze the content of your website and work with you to find out where you still have content potential and how you can leverage it. We also look at what content may even be hindering you and slowing down the rankings of your website overall.

There is always one word in focus here: relevance!

SEO quick check

At the beginning of most projects there is a quick check of your website. We check the technical and structural characteristics of your website. Afterwards, we discuss the results together and provide you with a prioritized task list. Of course, in the next step we will help you to implement the identified measures. We will support you as you need it. In consultation with you, we will either do it ourselves or work with your developers.

Competitive analysis

An SEO competitive analysis is a review of competitors' websites to determine what SEO techniques they are using and how successful they are. This can include, for example, checking keywords used by competitors, analyzing backlinks pointing to their websites, and comparing metrics such as domain authority and search engine ranking. An SEO competitive analysis can help improve one's SEO strategy and stand out among the competition.

Competitive analysis is always a double-edged sword. Of course you should know who your competitors are, what they are doing and what you can learn from them! On the other hand, your focus should be on making your website as good as possible for users.

We will gladly walk this fine line with you! We analyze your SEO competitors in terms of their structures and content and work out the most important characteristics for you.

What does an SEO competitive analysis include?

The exact scope of the competitive analysis depends, of course, primarily on your specific questions. As a rule, we analyze, among other things:

Your rankings compared to your competitors. Of course, we also look at whether your competitors cover interesting search queries that you do not yet serve.
We take a very detailed look at the webspeed and core web vitals of your site and your competitors. If desired, a continuous monitoring is developed from this. Often with individual page clusters down to single URLs.
We perform a basic audit of your website and the websites of your competitors. Here we look at the essential technical, structural and content aspects.

Monitoring

A one-time analysis is useful as a starting point. No question about it! However, you don't want to miss any significant changes and be overtaken on the left without realizing it. That's why we monitor your competitors on demand continuously with regard to their rankings, content, structure and technology.

What we can monitor for you, among other things:

Your own rankings based on Google Search Console (daily, weekly and monthly)
Your own SEO traffic development comprehensibly prepared based on Google Analytics (daily, weekly and monthly)
The visibility index of your site and your competitors based on SISTRIX (weekly)
Website structures and content of your site and your competitors. This corresponds to an SEO quick check that is carried out again and again (monthly).
Load times (web speed and performance, accessibility, best practices, Progressive Web App status and other SEO characteristics using Lighthouse (daily, weekly and monthly).
And apart from that, almost everything you can read out of a website. For example, price monitoring of competitors, user ratings and much more.

SEO sparring

As an SEO sparring partner, we support you or your SEO team in all questions and challenges around your website. As an experienced and external extension of your SEO team, we offer a fresh view on your topics and help you to get the best out of your website.

We know that with SEO you are never really "done". That's why we're happy to work with you over the long term to regularly review and improve your SEO strategy. With our regular reporting, you can track the progress of your rankings and achieve more visibility.

In addition to our SEO support, we also offer training so you and your team can use the best tools and techniques to optimize your website. From editorial training for hands-on content workshops to technical and analytics training for IT and Google tools, web analytics, reporting and dashboards, we've got you covered with everything you need for a successful SEO strategy.

SEO relaunch support

We support you with the relaunch or relocation of your website! As an agency and in-house SEOs, we have carried out numerous relaunches and learned a lot in the process. With our relaunch checklist, developed over the years, you have everything you need for a successful relaunch from an SEO point of view. Who wants to end up like that?

SISTRIX Relaunch frankfurt.de

According to an article by SISTRIX, this relaunch cost 1.4 million euros and was planned and implemented over eight years. Despite this, the site lost about 50 percent of its visibility and still hasn't recovered a year later. A particularly glaring example of a failed relaunch - admittedly.

Let's be honest: A relaunch is often simply stressful for many people involved. Expectations are high, and so are the costs. At the same time, technical resources often run out and people start to cut topics. From our experience often the wrong ones when it comes to SEO.

Profit from our experience and avoid mistakes that others have already made. We accompany your relaunch and lead it to success! And if the child has already fallen into the well, we help you to limit the damage and to bring your website back on the road to success.

Regular crawling of your website (and competitors)

We run our own crawler based on Screaming Frog! With it we can crawl your website regularly and cost-efficiently. We provide you with the data for your own analyses using Google Data Studio or discuss the results with you.

Our crawler can also render JS. Others have this option as well, but there it becomes very expensive very quickly.

FAQ about our crawler

: Your website can be crawled almost as often as you want. The most common intervals are monthly, weekly and daily. The limiting factor is the speed of your website and the number of pages, images, other content and API's used.
: Yes! The crawler renders JavaScript by default.
: With our crawler, almost all relevant SEO criteria can be queried and checked. During each crawl of your page, 150 to 200 properties per URL are checked, depending on the configuration and released data source. That is quite a lot! In addition, depending on the configuration and customer requirements, there are dedicated reports on technically more complex issues such as the validation of hreflang.

We can include numerous data sources in the crawl in addition to the crawl data of your own website. Provided you provide us with the appropriate login data and API keys. We can include the following data:

Google Analytics (Universal Analytics - access to your data required)

Google Search Console (access to your data required)

Pagespeed Insights (No access or API key required on your part)

Majestic (API key required on your part)

Ahrefs (API key required on your part)

Moz (API key required on your part)
: Our crawler is a Screaming Frog, which is operated in the cloud. The crawls are started automatically on a monthly, weekly or daily basis, depending on the settings. The crawling results are automatically written to Google Big Query and then queried or displayed via Google Data Studio.
: No, we do not offer this feature. The crawler is intended for continuous optimization of your website and continuous competition monitoring.
: Yes! We can also start the crawler manually on demand.