Access log ubuntu how to open. Linux system logs (logging management). Exim log files

The log file (log file), as you probably guessed, saves the chronology of events for the operating room Linux systems, its applications and services.

Files are stored in plain text to make them easy to read. In our article we will tell you where to find log files, give examples of several main log files and explain how to read them.

Where are the log files located in Linux?

Linux log files are typically stored in the /var/logs folder.

The folder stores a large number of files from which you can obtain information for each application.

For example, when the command ls is executed, for example, in the /var/logs folder, here are some available logs:

• kern.log
• auth.log
• bootstrap.log
• alternatives.log
• samba
• lightdm

The last three of this list are folders, but they also have log files.

Since log files are in text format, they can be read by entering the following command:

The above command opens the log file in nano editor. If the log file is small in size, then there is nothing better than opening the log file in an editor, but if the log file is large, then you are probably only interested in the end of the log.

You can specify how many lines to display using the -n option as follows:

If you want to see the beginning of the file, you can use the head command.

Basic system logs

The following log files are the main ones in Linux.

• Login log
• Demon Journal
• Debug log
• Kernel log
• System log

The daemon log (daemon.log) tracks services running in the background that perform important tasks. Demons are generally not shown on screen.

The debug log provides debug output for applications.

The kernel log contains detailed information about the Linux kernel.

The syslog contains most of the information about your system, and unless an application has its own log, the entries will likely be in this log file.

Analyzing Log File Contents

The image above shows the contents of the last 50 files in the system log file (syslog).

Each log line contains the following data:

• Hostname
• Service/Application
• Message

An example of one of the lines in the syslog file:

Jan 20 12:28:56 gary-virtualbox systemd: starting cups scheduler

Log rotation

The log files are changed periodically to prevent them from becoming too large.

The log rotate utility is responsible for rotating log files. You can tell that the log has been replaced because its name will contain a number such as auth.log.1, auth.log.2.

You can change the log rotation frequency by editing the /etc/logrotate.conf file.

Below is an example of entries from the logrotate.conf file:

#rotate log files
weekly

#keep 4 weeks worth of log files

create new log files after rotating

As you can see, these log files change every week, and four weeks' worth of log files can be stored at a time.

When a log file is replaced, a new file is created in its place.

Each application can have its own rotation policy. This is definitely useful because the syslog file will grow faster than the cups log file.

Rotation policies are stored in /etc/logrotate.d. Each application that requires its own rotation policy will have a configuration file stored in the application folder.

For example, the apt tool contains a file in the logrotate.d folder with the following content:

/var/log/apt/history.log (
rotate 12
monthly
compress
missingok
notifempty
}

Basically, this log tells you the following. The log will keep log files for 12 weeks and will change every month (one per month). The log file will be compressed. If no messages are written to the log (ie it is empty), this is normal. The log will not be replaced with a new one if it is empty.

To change a file's policy, replace its settings with the ones you want, and then run the following command:

Found a typo? Select the text and press Ctrl + Enter

Managing Log files on a Linux System

Linux's Log Files

All Linux systems generate systems logs that can be inspected to find information about your running system. These log files can contain a wealth of information from simple information messages to critical system issues. Most of the logging files that are created are in plain text. This means that it is very easy to view these using standard commands such as "more", "less", "cat", "head", "tail", "pg" or by using your preferred text editor such as "vi or vim".

By convention, most of the log files that are created are found under the directory "/var/log/". This is a standard area where system messages and logged/recorded. Depending on which Linux distribution you are using you will probably have a "message" file or a "syslog" file that contains recent activity. Logfiles are generally created by either a "syslogd" or "rsyslogd" logging demon. These demons are highly configurable and can filter messages into specified files. As well as handling local events, it is possible to log messages to remote servers dedicated to receiving these type of messages. It is quite common within larger organizations to have a dedicated syslog server. Some basic configuration options will be covered later. It is also common practice to have some form of log rotation process.

Below is a list of some of the more common log files that you will find. Some of these are distribution specific:

Although the above is not an exhaustive view of all the files that can be found within the "/var/log" area. It does give you a rough idea of ​​what is logged. It is important to remember that many third part products (software) will also write to this area. Often a sub-directory is created with various log information held within. Samba is a good example of this.

As mentioned earlier it is the "syslogd" or "rsyslogd" daemon that handles the majority of logging on your systems.

rsyslogd - logging utility

Rsyslogd is a reliable extended version of the syslogd service. Linux uses rsyslogd as its mechanism to record log files either in a central area or split into separate directories for clarity. It is also possible to send information to a dedicated logging server. Multiple processes may write to the same area without causing file locking. Simple commands can be used directly from scripts to write to this area.

Configuration Files

How rsyslogd behaves on your system is down to its configuration. This file is generally located in "/etc/rsyslog.conf". This file contains text which describes what should happen to messages when they are logged. It is here that you can specify specific directories for specific message types. Default logging rules are generally located under "/etc/rsyslog.d/"

Example of rsyslogd.conf under Ubuntu

/etc/rsyslog.conf

Hashes "#" are used to denote a comment or for commenting out a function that is not required.
Notice the last line $IncludeConfig /etc/rsyslog.d/*.conf. This is where we can specify custom rules/mappings.

The default logging area is called "syslog", see below exert

What are Facilities and Levels?

Whenever the rsyslogd daemon receives a logging message, it acts based on the message type (Facility) and a Level (Priority). These mappings can be seen in your "/etc/syslog.conf" file or your included "/etc/syslog.d/*.conf" files.

Each entry within the configuration file can specify one or more facility/level selectors followed by an action. A selector consists of a facility or facilities followed by a single action. Action is normally the name of the directory and file that is to receive the messages into.

facility.level action

Example: mail.* -/var/log/mail- Here, "mail" is the facility, level is set to "*" and action is "/var/log/mail"

Facility

A facility represents the creator of the message, these normally consist of:

auth, authpriv, cron, daemon, kern, lpr, mail, mark, news, syslog, user, local0 - local7, " * " signifies any facility

These facilities give us the ability to control where messages from certain sources are sent to. The facilities local0 - local7 are for use by your own scripts.

Level (Priority)

The level specifies the severity threshold. These can be: (lowest priority first)

debug, info, notice, warning, err, crit, alert, emerg.

On older systems you may see "warn, error or panic". A level of none will disable the associated facility. These priorities control the amount of detail that is sent to each logfile. A single period "." separates the facility from the level. Together these are known as the message selector . An asterisk" * " may be used to specify all facilities or levels. Similar to facilities, wildcards "*" can be used along with "none". Only one level or wildcard may be specified with each selector. The following modifiers may be used " = " and " ! "

If you specify only one level within a selector without any modifiers, you are actually specifying that level plus all other priorities. For example the selector user.notice is actually saying all user related messages having a level of notice or higher will be sent to the specified action area. If you require only a level of "notice", then you will have to use the " = " modifier:

user.=notice- Now means any user related messages with a level priority of "notice" only will be sent to the relevant logging area.

If you use the " ! " modifier, this will negate your level priority. So if we specified user.!notice is the equivalent of all user related messages with a level priority of "notice" or higher. You can also specify user.!=notice which specify all user related messages except for those with the level priority of "notice".

Actions

The action section is the destination for the messages. The action can be a filename such as "/var/log/syslog" or a hostname or IP address prefixed with the "@" sign. The latter option is popular in large organizations and enterprises. Quite often security related messages may be sent to a central logging server for further investigation.

rsyslog.conf structures

As rsyslogd is an enhanced version of the syslogd it can handle the older legacy style constructs known as sysklogd. It also handles legacy versions of rsyslog. However, the true power of rsyslog comes into play when you use what is known as " RainerScript". This is the new style format for rsyslog which can handle complex cases with ease. In the example below you can see old format entries along with newer entries that use "if - then" constructs for a more precise handling.

Example section of "/etc/rsyslog.conf taken from openSUSE

Message Testing with the logger command

logger is a shell command interface into the syslog module. Logger allows you to make entries directly into the system log. This is very useful when incorporated into a script or when you want to test your message selector and mappings.

In its simplest form we can issue logger "I am a test". This message would then go to our default area (probably /var/log/syslog or /var/log/messages) depending on how you have configured your rules. You can also specify a priority using the "-p or --priority" option. Examples of logger action:

Basic Logger Usage

Getting help with Rsyslog

The above is intended as an overview to the processes that takes place for logging of messages to occur. For further information you can issue "man rsyslogd" from your console for an overview of the many options. For further reading you can head to the main "rsyslog" website: www.rsyslog.com

dmesg

"dmesg" is a special command that stands for display message. dmesg will display the message buffer of the kernel. dmesg is very useful if you want to view the messages that flew past your screen quickly during the boot process. Another useful trick is to redirect the output from the dmesg command to a temporary file: dmesg > /tmp/temp.txt.

dmesg is also useful if you are having issues with an I/O device or a "USB" device. dmesg can be used in combination with the grep command to find exactly what you are looking for quickly: dmesg | grep -i usb

dmesg basic overview

As you know, Linux has a very important tool sysklogd, which allows you to keep a log of events occurring in the system. System administrators, and any user who encounters errors will have to look linux logs to identify the problem and solve it.

All linux log files are located in the folder:
/var/log

You can view Linux logs using several console programs. Below we will look at a few examples:

1. Outputting and scrolling through text using less:
less /var/log/messages

2. View logs in real time:
tail -f /var/log/messages

3. Open the file using cat:
cat /var/log/dmesg

4. Output the first 10 lines from the file:
head /var/log/dmesg

5. Output the last 10 lines from the file:
tail /var/log/dmesg

6. Output a certain number of lines:
head -n3 /var/log/dmesg
where -n3 is the number of lines to be shown.

7. We display only errors:
grep -i error /var/log/messages

In addition to the console and text editors you can also use graphics program“System Log Viewer”, which will show you in a convenient and visual form Linux logs.

When viewing log files (log files) in Linux, sometimes you need to monitor new log entries in real time. That is, you specify which log file (or files) you want to view and monitor in real time for new entries in that file.

Monitoring log files with the tail command

If you run the command tailFileName without any additional arguments, 10 will be printed last lines file and the command will complete its work.

In order for the tail command to continuously output the last notes in the file, that is, if new entries appeared in the file, then the information on the screen was updated, the -f option is used:
tail -f LogFileName

Run the tail -f command to output the log file /var/log/syslog

tail -f /var/log/syslog

Since the option is used -f , the tail command does not complete its work, but waits for new entries to appear in the log file. As soon as new entries are added to the log file, they will immediately be displayed in the terminal.

To interrupt the execution of a command, press the keyboard shortcut Ctrl+C

tailf command

The tail -f command is equivalent to the tailf command.

Usage:

Tailf /var/log/mylogfile.log

An important difference between the tailf command and tail -f is that tailf does not access the file when it is not modified. As a consequence, the file access time is not updated and the system does not continuously flush the file to disk when the file is not updated.

The description of the tailf command indicates that it is useful for monitoring log files on laptops. Since the disk is not accessed unnecessarily and battery life is preserved.

tail -F. If the file has been renamed or deleted

Typically, in Linux, log files are not written indefinitely, otherwise such a file would be very inconvenient to use in the future. Instead, so-called file rotation is used. When the log file becomes large, it is either deleted or renamed (created backup copy file), and further messages begin to be written to a new, empty file.

The tail command has two options: -f and -F

• If used option -f and the tracked file is renamed, the tail command continues to track the already renamed file. Team tail in this case it is tied to the identifier (inode) of the file.
• If used option -F and the tracked file is renamed, the tail command will detect this, and as soon as a new log file is created (with the name that we specified to the command tail), team tail will start tracking this new file.

Let's look at an example.

We will monitor the log file /var/log/apache2/error.log. Execute the tail command with the -F option

Tail -F /var/log/apache2/error.log

If the system moves (renames) the error.log file to error.log.1 and creates a new error.log file, then our tail command will continue to monitor the new error.log file

If we were to use the option in this example -f , then the tail command would continue to monitor the error.log.1 file, which is no longer relevant for us when viewing logs in real time.

Monitor multiple log files simultaneously

The tail command supports tracking multiple files at once. To do this, you must specify file names separated by spaces.

Tail -f /var/log/apache2/error.log /var/log/apache2/access.log

As soon as any of the files changes, the name of this file and the new entries in it are displayed on the screen.

For simultaneous monitoring of several log files, there is a very convenient multitail utility

It does not just display data, but creates its own window (area) for each file and displays the data in this window. With its help, it is very convenient to track many log files at once and see them in one terminal window.

The multitail utility can be installed from the standard repositories of your distribution. To install, run the command (select the appropriate command for your distribution):

Sudo apt install multitail sudo yum install multitail sudo dnf install multitail

Usage:

Multitail /var/log/apache2/access.log /var/log/apache2/error.log

To exit the utility, press the q key

It is impossible to imagine the user and administrator of the server, or even workstation on Linux based, who never read the log files. The operating system and running applications constantly create Various types messages that are logged in various log files. Ability to determine required file log and what to look for in it will help to significantly save time and quickly resolve the error.

Logging is the main source of information about system operation and errors. In this short guide, we will look at the basic aspects of operating system logging, directory structure, and programs for reading and reviewing logs.

Basic log files

All log files can be classified into one of the following categories:

• applications;
• events;
• services;
• systemic.

Most log files are contained in the /var/log directory.

• /var/log/syslog or /var/log/messages contains a global system log in which messages are written from the moment the system starts, from the Linux kernel, various services, detected devices, network interfaces and much more.
• /var/log/auth.log or /var/log/secure- information about user authorization, including successful and unsuccessful login attempts, as well as the authentication mechanisms involved.
• /var/log/dmesg- device drivers. Using the command of the same name, you can view the output of the file contents. The log size is limited, when the file reaches its limit, older messages will be overwritten by newer ones. By setting the --level= switch, you can filter the output by significance criterion.

• /var/log/alternatives.log- The output of update-alternatives, which contains symbolic links to the default commands or libraries.
• /var/log/anaconda.log- Entries recorded during system installation.
• /var/log/audit- Records generated by the auditd audit service.
• /var/log/boot.log- Information that is written when the operating system boots.
• /var/log/cron- Crond service report executable commands and messages from the teams themselves.
• /var/log/cups- Everything related to printing and printers.
• /var/log/faillog- Failed login attempts. Very useful when checking security threats, hacker attacks, and brute force hacking attempts. You can read the contents using the faillog command.
• var/log/kern.log- The log contains messages from the kernel and warnings that can be useful in troubleshooting errors in custom modules built into the kernel.
• /var/log/maillog/ or /var/log/mail.log- Magazine mail server, used on the OS.
• /var/log/pm-powersave.log- Battery saving service messages.
• /var/log/samba/- Samba file server logs, which is used to access shared Windows folders and providing access Windows users to Linux shared folders.
• /var/log/spooler- For the old school, contains USENET messages. Most often it is empty and abandoned.
• /var/log/Xorg.0.log- X server logs. Most often they are useless, but if they contain lines starting with EE, then you should pay attention to them.

There will be a separate package manager log for each distribution.

• /var/log/yum.log- For programs installed using Yum on RedHat Linux.
• /var/log/emerge.log- For ebuilds installed from Portage using emerge on Gentoo Linux.
• /var/log/dpkg.log- For programs installed using dpkg on Debian Linux and the entire family of related distributions.

And some binary logs of user sessions.

• /var/log/lastlog- Last user session. You can read it with the last command.
• /var/log/tallylog- Audit of failed login attempts. Output to the screen using the pam_tally2 utility.
• /var/log/btmp- One more log of unsuccessful login attempts. Just like that, just in case you haven’t figured out where to look for traces of hacker activity.
• /var/log/utmp- List of currently logged-in users.
• /var/log/wtmp- Another log to record user logins. Displayed on screen using the utmpdump command.

And other magazines

Since an operating system, even one as wonderful as Linux, in itself does not provide any tangible benefits, most likely a database, a web server, and various applications will be running on a server or workstation. Each application or service can have its own event and error log file or directory. Naturally, it is impossible to list all of them, only a few.

• /var/log/mysql/- MySQL database log.
• /var/log/httpd/ or /var/log/apache2/- Web log Apache server, the access log is in access_log and errors are in error_log .
• /var/log/lighthttpd/- Lighttpd web server log.

The user's home directory may contain logs graphic applications, DE.

• ~/.xsession-errors- Stderr output from X11 graphics applications.

• ~/.xfce4-session.verbose-log- XFCE4 desktop messages.

What to view - lnav

Almost everyone knows about the less utility and the tail -f command. The vim editor and the Midnight Commander file manager are also suitable for these purposes. They all have their drawbacks: less doesn't handle logs well with long lines, taking them for binaries. Midnight Commander is only good for quick browsing when there is no need to search through a complex pattern and go back and forth a lot between matches. The vim editor understands and highlights the syntax of many formats, but if the log is updated frequently, distracting messages about changes to the file appear. However, this can easily be circumvented using<:view /path/to/file> .

Recently I discovered another useful and promising, but still slightly damp, utility - lnav, as transcribed by Log File Navigator.

Install the package as usual with one command.

Log Navigator lnav understands a range of file formats.

• Access_log web server.
• CUPS page_log
• Syslog
• dpkg.log
• strace
• Custom entries with timestamps
• gzip, bzip
• VMWare ESXi/vCenter Log

What does understanding file formats mean in this case? The trick is that lnav more than a utility for viewing text files. The program can do something else. You can open multiple files at once and switch between them.

The program can directly open an archive file.

Shows a histogram of informative messages, warnings and errors when you press a key . This is from my syslog.