The logs are text files with very useful information to identify problems and errors. They are commonly used in computing to search for bugs or unwanted behavior of computer applications, mobile apps, or websites.
The log analysis is a very useful process for search engine optimization since they allow you to consult the information about which robot has accessed the web, which URL has been crawled, at what time it has occurred access, and other data of interest to improve SEO.
What is a log?
A log is a text file that collects chronologically (ordered in time) the incidents that have been affecting an application or computer system, as well as their changes during this process.
The main use of a log is to identify errors , serving as a very valuable source of information on the behavior of web or computer applications.
For example, if there is a problem with a website created with the WordPress CMS, you can easily check the WordPress error logs to find where, when and what is causing the error.
The usual thing is that a daily log of a website is generated
Data I can get from log analysis
The analysis of the log files allows obtaining important information about access to a web or web server. The main data that can be obtained from log analysis and that is very useful for SEO are:
- The access IP address, whether it is a visit or a search engine bot.
- The name of the host.
The country or region of origin. - The browser and operating system are used to access the web.
- The page where access leaves the website.
- User-agent where the bot that has made the access is identified.
- The type of access that has been made (direct access by the user or through a reference from another web page, social network, or advertising medium).
- The time spent on the page and the number of pages you have browsed.
Where can I see the log file of a website?
The logs of a website are usually stored in the logs folder, so it is necessary to access the webserver to be able to view its content.
The process to open a website log can be done in two different ways:
- In the options of the administration panel of the contracted hosting, access the administration or file management option, from which you can navigate through the different folders that the web page contains to locate the logs folder and the log files.
- Using an FTP application to directly access files from the web. Filezilla is a great alternative to access via FTP since it is free and very easy to use.
When is important to do a log analysis?
Log analysis is a technical SEO practice that is very important to complement the information available on the behavior of users and bots from Google and other search engines.
Although this type of analysis is essential for large web pages with a high volume of visits, can be applied to all types of websites because many benefits can be obtained.
Advantages of performing a log analysis
Log analysis is a great way to complement the analysis of visits of a web page, eCommerce, or blog. Complemented with the use of other tools such as Google Analytics or SEMrush, it allows to better understand the behavior of users and Google robots that access the web.
Optimize the crawl budget
With the log, analysis is possible to determine the crawl budget, or time spent by Google robots to crawl and know the pages of a website. In the logs, you can determine which pages Google is crawling on the website, which ones it spends more time on, and which ones it does not access or hardly spends time on.
With this information, it is possible to apply SEO measures and techniques on-page to make sure that the bots crawl the most interesting pages, to achieve better positions in the search results pages or SERPs.
With the log analysis, it will be possible to identify if Google bots spend more time on the pages to be positioned, if they lose crawling time on error or low-value pages, and ultimately if the the positioning strategy that is being carried out is on the right track.
File download and access control
Through the log, you can control and know access to the web and its files. If you have files that users can download, it is a good way to know how many downloads have been made, who has downloaded it, and other relevant information about it.
Access to statistical data
The logs are usually saved in files by dates, which allows access to different logs with information on time periods. The analysis of all these records allows to obtain statistical data that allows evaluating the web over time .
It is easy to analyze
Accessing and analyzing a log is not a complex process since it is only necessary to access the file and review its content. It will not be necessary to use cookies, programming languages such as JavaScript, or embedded HTML code.
Export to other tools
In the case of logs that are not too large it is useful to export them to other tools that facilitate their organization and reading . For example, using the Excel spreadsheet, log files can be imported to process the data using its tables, filters, formulas, search system and other tools.
Avoid problems with user data protection
Experts in SEO and log analysis
Log analysis tools
A lot of information in text format is concentrated on one line of a log, so using specific tools speeds up and facilitates the process of understanding the information it provides. This type of data visualization program allows you to identify all the information and offer it in a much easier format for understanding.
Among the different tools that can be used for the analysis of records, we can classify them as:
Applications that connect to the server
This type of tool connects to the server (granting it the corresponding permissions for this) and directly accesses the log files to extract the information and present it in a more attractive and easy-to-understand visual format.
Among this type of software, the following stand out:
- Fandango SEO. This tool created by an SEO expert works in the cloud and connects directly to the server to access the logs that are generated. It presents the information segmented with bar graphs and other very interesting options.
- Seolyzer. It is an application that automates the analysis of logs and is very interesting as it is a freemium application that allows you to consult a lot of information about the logs without having to go to the paid version.
Applications that import logs
In order to access the information on the logs with this type of tool, it is necessary to download the logs and dump them on the tool. Although they do not use an automated process like those that connect directly to the server, there are very powerful and useful tools for log analysis such as:
- Screaming Frog Log Analyzer . Perhaps one of the best log analysis tools, it has multiple options such as the ability to view bots by type (bingbot, googlebot, yahoobot …), choose analysis by date range, or verify which bots are authentic and which are not. . In addition to presenting the information in the logs in a table format and well organized, it also incorporates graphical presentations that make it much easier to understand the information at a glance.
- SEMrush Log Analyzer . It is a tool incorporated in the SEMrush suite and it is also very useful and graphical when it comes to displaying the information contained in the logs.
