FGT1 02 Logging and Monitoring V2

In this lesson, we will look at how to monitor your FortiGate, and how to log its system events andnetwork traffic. Since you are implementing a security solution, it is important to know how toappropriately monitor the device’s operation. It is vital to have logging and monitoring configuredproperly and to know how to read the output. Otherwise if you encounter issues, you won’t have anymessages from FortiGate to help you find out what is happening in your network.

Logging & MonitoringDO NOT REPRINT© FORTINET

By the end of this lesson, you’ll be able to:Describe log severity levelsIdentify where logs are storedDescribe the different types of logsUnderstand log structure and behaviorConfigure log settingsUnderstand the impact of logs on resourcesDescribe how to view log messages, and finallyDescribe how to search and interpret log message


The basic purpose of logs is to help you monitor your network traffic levels, track down problems,establish baselines and a lot more.

Think of your own internal organization, where it is highly probable that more than one administratorhas access to your FortiGate device. Since it is not practical to block other administrators from makingchanges to your FortiGate configuration, you can simply view the log files to find out what ishappening on the device—including any changes that were made. Logs help provide you with the bigpicture so you can make adjustments to your network security, if necessary.

Keep in mind that some organizations have legal requirements when it comes to logging, so it isimportant to be aware of your organization’s policies during configuration.


Each log entry includes a log level that ranges in order of importance from Debug to Emergency. Intotal there are eight levels. Debug, the lowest level, puts additional information into the event log andis worthless unless you are actively investigating something. Debug is only needed to log diagnosticdata, puts more strain on the CPU resources, and requires additional resources to create. Generallythe lowest level you want to use is Information.

You and your organization’s policies dictate what needs to be logged.


You can choose to store logs in a variety of places both on and off the device. Locally, the FortiGatedevice has memory and many devices have a built-in hard drive. Externally, you can store logs onSyslog Servers, FortiCloud, SNMP, or a FortiAnanlyzer device.


As an external logging device for FortiGate, a FortiAnalyzer or FortiManager is simply viewed as an IPwith which the FortiGate can communicate. As a result, you can place a FortiAnalyzer orFortiManager within the same network as a FortiGate, or outside of it. However, a Fortigate cancommunicate with a FortiAnalyzer or FortiManager only if it is registered device. So long as theFortiGate is properly registered with the FortiAnalyzer or FortiManager, it accepts incoming logs.Communication between the Fortigate and FortiAnalyzer or FortiManager is done via SSL encryptedOFTP traffic, so when a log message is generated, it can be safely transmitted across an unsecurenetwork.


So far, we’ve discussed FortiAnalyzer and FortiManager as interchangeable external logging devicesfor the FortiGate. While configuring the FortiGate to send logs to a FortiAnalyzer or FortiGate isidentical—they share a common hardware and software platform—the FortiAnalyzer andFortiManager actually have different capabilities that are worth noting. Both take log entries, but aFortiManager’s primary purpose is to centrally manage multiple FortiGate devices. As such, it has aflat limit imposed on the amount of logs it can receive in a day, regardless of the model. On the otherhand, the FortiAnalyzer’s primary purpose is to store and analyze logs, so the log limit is much higher(though the limit is model-dependent). Even the smallest FortiAnalyzer can handle more logs per daythan any FortiManager.

But at the most basic level, what you can do with the logs received on a FortiManager is no differentthan what you can do with logs received on a FortiAnalyzer.

The FortiGate has 2 methods for transmitting the log events. There is the store-and-upload option, aswell as real time.


You can configure logging to either a FortiAnalyzer or FortiManager through the GUI or CLI.

In the GUI, it is done under Log & Report > Log Config > Log Settings. Here, each device must be setup separately, one at a time.

In the CLI, you can configure up to three separate FortiAnalyzer or FortiManager devices at the sametime. The options in the GUI only relate to the ‘config log fortianalyzer setting’, not fortianalyzer2 orfortianalyzer3. You may need a setup like this for redundancy or for some other requirement. Keep inmind that generating logs requires resources, so the impact of sending logs to multiple locationsultimately depends on how many logs you are creating.


Another external logging option you can use is FortiCloud. FortiCloud is a subscription-based service,offered by Fortinet, that offers long term storage of logs as well as provides reporting functionality. It’sa similar idea to FortiAnalyzer, but more advantageous for smaller setups, where purchasing adedicated logging appliance isn’t feasible. Every FortiGate comes with a free one month trial. You canactivate your free trial from the GUI and link it to your FortiCare user and start sending logs. Be sure toread any documentation on the website if you are considering the subscription-based option.


On the FortiGate, all logs are split up into three different log types. These are traffic logs, event logs, andsecurity logs.

Each log type is further split up into sub-types. Traffic logs contain Forward, Local, Invalid and Multicast.The Forward log contains information about traffic either accepted or rejected by a firewall policy. Localtraffic is traffic directly to/from the FortiGate, and includes logging into the GUI, as well as FortiGuardqueries. Invalid packets are the logs thrown away before they even get to a firewall policy.

Event logs contain System, User, and Router/VPN/WanOpt &Cache/Wifi sub-types. System events arerelated to system operations, such as automatic updates of the AV/IPS definitions and people logginginto the GUI. User contains logon/off events for users hitting firewall policies. Router/VPN/WanOpt&Cache/Wifi contain log entries related to the specific feature. For example, Router contains BGP orRIP log entries and VPN contains IPSec and SSLVPN log entries.

Finally, Security logs contain log entries based on the security profile type. For example, Antivirus, WebFilter, and Intrusion Protection to name a few. Security logs only show specific sub-types if logs arecreated within it.


The Log & Report section of the FortiGate GUI includes the three log types: Traffic, Event, and (ifconfigured), Security. The Traffic Log contains events about packets. The Event Log contains admin orsystem activity events. The Security Log contains messages related to security profiles activated onfirewall policies. By default, most of the events related to security appear in the Forward Traffic log—asub-type of the Traffic Log. This is for performance: fewer log files is less CPU intensive. The exceptionto this is DLP and Intrusion Scanning. Events such as these always appear in the Security Log section.


To inspect your logs through the GUI, go to the Log & Report section and select the log type to view.In the upper right corner of the window, you can switch between viewing the logs from differentlocations if the FortiGate is set up to log to multiple locations.

It is not recommended to configure your firewall to actively inspect traffic without creating a log entryabout it.


This chart illustrates the expected behavior when you enable different logging options.

The first column, Policy Log Setting, shows the log setting on the Firewall policy: No Log, Log SecurityEvents, or Log all Sessions.

The second column shows whether an Antivirus, Web Filter, or Email security profile is enabled ordisabled. Remember, DLP and IPS profiles always generate logs in the Security Log section.

The last column shows the behavior. If you enable any profiles on your policy and logging is not enabled,you will not get logs of any kind—even if the profile is configured to block the traffic. So if you apply asecurity profile, it’s important to remember to consider the logging setting.


When viewing the logs, you might encounter a high volume of log messages, depending on yourconfiguration. This makes it difficult to locate a specific log or log type, especially during aninvestigation. In order to negotiate the logs more efficiently, you can set up various filters. The moreinformation you specify in the filter, the easier it is to find the precise log entry. Filters are configuredfor each column of data you choose to display. By default only a subset of the information appears inthe log table. Make sure to configure the table columns for your own requirements.


Every log message you view has a standard layout comprised of two sections: a header and a body.The header contains the same information regardless of the log. The body, however, changes fromone type of log message to another. This is because there is some data common to all logs, like adate and time, while other data is event dependent.


Let’s take a closer look at the header in this is an example of a raw log entry. While the output is notas structured as it appears in the GUI, the information contained in a raw log file is the same. As youcan see in the header, aside from the date, time, and log ID attributes, you can see the that log type isUTM, the sub-type is DLP, and the severity level is Warning. The attributes in the header (such as logtype and sub-type) are common to every log, but the data aligned to it can be different. For example,the header can contain a log type of Event and sub-type of System instead of what you see in theexample above. Accordingly, the information in the header of the log directly effects the datacontained in the associated body of the log.

Note that if you log to a 3rd party device, such as a Syslog server, you need to know how to set upyour filters in order to find what you need in your log messages. You can find a document thatcontains all the logs and their layouts from the Fortinet docs web site at http://docs.fortinet.com .


Now lets take a closer look at the body of a log. The body provides the specifics of the log messageand helps you understand what actually happened. In the above log, we can see the action taken bythe FortiGate device when it encountered the traffic through the status attribute. Here, the status isDeny, which means the FortiGate prevented this particular piece of traffic from passing. The valueindicated by policyid field provides useful information about the policy this traffic passed through(which firewall rule was used).


Rather than look at raw logs or logs through the GUI, you can also display log messages from the CLI.This allows you to set up a number of filters on the logs that display and capture the output to a fileand send it via the options you specify, such as FTP.


Monitoring your logs is critical, as it allows you to review the progress of an attack, whether afterwardsorwhile in progress, and address the issue quickly. How the attack unfolds may reveal weaknesses inyour preparations.

There are three ways you can monitor logs: Alert Emails, Alert Message Console, and SNMP.


Since you can’t always be physically at the device, you can monitor logs by setting up Alert emails.Alert emails are set up similar to any log device. First you decide “what” is going in to them (a filter)and then “where” it is going.


In order to set up an alert email, the first thing you need to do is configure an SMTP server to allow forcommunication between the server and the FortiGate device. This can only be done in the CLI.This allows you to configure your alert email settings in the GUI through the Log & Report > LogConfig > Alert E-mail menu. Without configuring an SMTP server that will receive the email, the alertemail option does not appear in the GUI.


Another log monitoring option is the alert message console. The Alert Message Console is a GUIwidget that you can enable on the System dashboard. Here, instead of the alerts being emailed toadministrators like in Alert emails, they appear directly in the widget on the System page when you login to the FortiGate. You can configure the widget to set up the events you want to appear as alerts, thenumber of alerts, and even the name of the widget itself. For example, you can have multiple alertwidgets on the dashboard with different names all displaying different types of alerts.

Once an alert appears in the Alert Message Console it remains until acknowledged. Once you confirmthe event did not impact anything, you acknowledge it, and it is removed from your list — it no longerappears as something that requires further attention.


Another method of monitoring logs is through an SNMP manager. In order to use this method, yourequire the Management Information Base (MIB) file. A MIB is a text file that describes a list of SNMPdata objects that are used by the SNMP manager. These MIBs provide information the SNMPmanager needs to interpret the SNMP trap, event, and query messages sent by the FortiGate deviceSNMP agent. They can be loaded into any SNMP software so that you can set up automatic queriesto the device in order to discover operational status. You can obtain CPU, memory levels, the causefor the last spam detection, and more. A FortiGate device can support SNMP v1, v2 and v3.

You can obtain the MIB files either on the Support website or directly from the FortiGate GUI throughthe System > Config > SNMP menu.


Setting up the necessary SNMP options is fairly straight forward from the GUI. Simply enable anddefine the service as you would any other SNMP monitored device and then enable your protocoloptions and methods of monitoring. What can be monitored with the different options is exactly thesame. SNMP v3 offers some additional security over the previous two versions of the protocol, liketraffic encryption and authentication.


In the GUI, under Log & Report > Log Config > Log Settings, you can enable different locations for logstorage. You can also configure the different kind of traffic you want to appear in the Local traffic log.Finally, you can configure the GUI preferences. Resolving IPs to host names requires the FortiGate toperform DNS lookups for all the IPs. If your DNS is not working or running slowly, this can impact yourability to look through the logs as the requests will timeout.


Using the CLI to configure log settings provides you with more flexibility and options than the GUI.From the CLI, you can configure up to three separate FortiAnalyzers and Syslog servers, options notavailable in the GUI. There is also the ability to set up logging to Webtrends, a 3rd party service. Theinformation you require for configuring the log settings is dependent on the logging option youconfigure: disk, FortiAnalyzer, FortiGuard, memory, Syslog, or Webtrends.


Firewall policies also have logging options you can configure. The policy setting determines if andwhen a log message is generated for traffic passing through a particular firewall policy. The settingsunder Log Settings in the GUI and the ‘config log’ command in the CLI determine where the FortiGatestores the log messages it creates.


It’s important to remember that creating logs is not “free”—it does weigh on your system. The morelogs that get generated, the heavier the toll on your CPU and memory resources. Storing logs for aperiod of time also requires disk space, as does accessing them. So before configuring logging, makesure its worth the extra resources and that your system can handle the influx.

Also important to note is logging behavior with UTM profiles. UTM profiles create log events whentraffic is detected. Depending on the amount of traffic you have and logging settings that are enabled,your traffic logs can easily become a problem that will ultimately impact the performance of yourfirewall.

There is an option in the CLI that removes some of the information stored in the traffic log: set brief-traffic-format enabled. By executing this command, you can free up resources on the firewall.


In configuring the Event log settings, remember that Event logs are not caused by traffic passingthrough firewall policies. For example, VPNs going up and down or routing protocol activity are notcaused by traffic passing through a firewall policy. One exception might be the user log. This does notrecord information about traffic through firewall policies directly, but it does record user logon/logoffevents on traffic that passes through policies.

Event logs provide all of the system information generated by the FortiGate device, such asadministrator logins, configuration changes made by administrators, user activity, and daily operationsof the device. So what you enable depends on what features you are implementing and whatinformation you need to get out of the logs. You can enable what events you want to log through theLog & Report > Log Config > Log Settings menu.


There is also a daily log monitor section. This displays the number of logs generated over time as wellas the log type. This allows you to see where your FortiGate device is using most of its resources andif any trends are occurring. You can drill down through these logs and obtain further information byclicking any of the days.


Each function of the FortiGate device has an equivalent “Monitor” menu item in the GUI. This allowsyou to take a view, at any given moment, how the feature is performing. The Security functions have amonitor option like the rest, but you need to enable it from the CLI before it appears. With a lot ofsecurity activity this could impact your CPU, so it’s disabled by default.


One example of a GUI monitor is the Security Profiles monitor, found in the GUI under SecurityProfiles > Monitor. It has sub-sections for each security feature to highlight recent activity, such as AVMonitor, Web Monitor, and Application Monitor to name a few. This gives you a snapshot of what ishappening with that particular option. Almost every menu has this option.


Another means of monitoring is through the widgets on the status page. Many can be customized toshow the same type of information in multiple ways. If you click the pencil icon in the upper rightcorner of the widget, you can configure any of the available settings for that widget. You can add somewidgets to the same dashboard multiple times, with each instance displaying different information.


By default, there are a number of different dashboards available. Each one has a different name with adifferent collection of widgets to provide different types of information. Each user has their owndashboard setup and layout, so if one user deletes a dashboard and rearranges the widgets on theStatus page, it will not impact any of the other users. You can alter a user’s permissions to not allowthem to make changes to their dashboard and use this to restrict their access.


One other area you may want to monitor, purely for diagnostics, is the crash logs, available throughthe CLI. The FortiGate is like a computer, with different processes that handle different things, likeDHCP or web filtering for example. Any time a process is closed for any reason, the crash log recordsthis as a crash. If there is an abnormal termination of a process, you can look at the crash logs andfind out the conditions that caused it. A normal and fairly common thing to see in the crash log areentries for Scanunitd, which is the process responsible for virus scanning. Any time the definitionspackage is updated, that process needs to close down in order to apply the new package. This is anormal shutdown and appears with a status of zero, which indicates a normal shut down with noabnormalities.


In this lesson, we covered log severity levels; storage locations; log types and subtypes; log structureand behavior; log settings; viewing logs messages; and monitoring, reading, and interpreting logmessages.


Documents

FGT1 02 Logging and Monitoring V2