What Do u Mean by This Define Log_local 5 (21

Instructor: Dr. T.Y. Wong

Open-Source Software Project Development

Week 11, part 3Writing Realistic Server Software (5)

- Logging and Miscellaneous Topics

Spring semester2008-2009

CSC4140 – Open-Source Software Project Development

Page 2

A miscellaneous,but important topic missed

-How to close unnecessary file descriptors?



Page 3

Back to our previous scenario…

apache2[main thread]

server

created by more than one fork()

We know that there are many things inherited from the apache2 thread…

E.g., opened files…



Page 4

Closing unwanted files…• Why is it important?

– Because you would never know when the unwanted, opened files brought you troubles!

Process A

Sorry, I only have an inappropriate example…

Process BProcess C

PIPE

Someone forgets closing it…

Although this end of pipe is closed, the other end cannot find the pipe is close…



Page 5

OK, let’s KO them, but…• Question (1): how can we know if a file is

opened?– Sorry, there is no direct system calls…– We need to use some indirect ways…

int is_opened(int fd) {struct stat st;if(fstat(fd, &st) == -1)

return 0;else

return 1;}

int is_opened(int fd) {if(fcntl(fd, F_GETFL) == -1)

return 0;else

return 1;}

When an error occurs, it means the file descriptor is not associated with any opened files.



Page 6

OK, let’s KO them, but…• Question (2): how many file descriptors do

we have to try?– This time we have a direct system call!

int max = sysconf(_SC_OPEN_MAX);

Constant inside the system

_SC_OPEN_MAX Maximum number of opened files

_SC_CLK_TCK Clock ticks per second

_SC_ARG_MAX Max number of arguments

_SC_CHILD_MAX Max number of processes per users.



Page 7

Now, we can KO them eventually…

int is_opened(int fd) {if(fcntl(fd, F_GETFL) == -1)

return 0;else

return 1;}

int main(void) {int i, max = sysconf(_SC_OPEN_MAX);for(i = 0; i < max; i++) {

if(is_opened(i))close(i);

}......

}

This should be 3 instead of 0…

Any Question?



Page 8

Logging through syslogd…



Page 9

Why using syslogd…

Provides a centralized control over all the logging files.Pros

Provides a single service that serves all the processes in the system.

Single point of failure. What if it suffers a denial-of-service attack?

Cons

Effort is required when extracting information out of the log files; a user process is not likely to have its own log file.



Page 10

Logging using syslogd

syslogdUDP only

Process

Process

UNIX-DomainSocket

InternetSocket

/dev/log

port 514

Anyway, syslogd is a daemon that accepts UDP requests.

The requests contain the location, the level, and the content of the log entry.

The same machine



Page 11

BTW…• What is a UNIX-domain socket?

srw-rw-rw- 1 root root 0 2009-03-26 02:17 /dev/logsocket flag!

Unix-Domain Socket Internet Socket

socket(AF_LOCAL, SOCK_STREAM, 0);socket(AF_LOCAL, SOCK_DGRAM, 0);

socket(AF_INET, SOCK_STREAM, 0);socket(AF_INET, SOCK_DGRAM, 0);

bind(): target is a pathname;using struct sockaddr_un;

addr.sun_family = AF_LOCAL;strcpy(addr.sun_path, “hello.txt”);

bind(): target is IP addr & port number;using struct sockaddr_in;

addr.sin_family = AF_INET;addr.sin_addr…addr.sin_port...



Page 12

Architecture of syslogd…

/dev/klogUDP port

514

kernelroutines

/dev/log

TCP/IPNetwork

LocalProcess

syslogd

messages

auth

debug mail

Inside "/var/log"…The configuration file "/etc/syslog.conf" tells the daemon how the log files are used…



Page 13

Configuring syslogd…• Remember, you have to become "root" in order to

configure…

auth,authpriv.* /var/log/auth.log*.*;auth,authpriv.none -/var/log/syslog#cron.* /var/log/cron.logdaemon.* -/var/log/daemon.logkern.* -/var/log/kern.loglpr.* -/var/log/lpr.logmail.* -/var/log/mail.loguser.* -/var/log/user.log

The pathnames that the log messages to be stored.Snippet from "/etc/syslog.conf"

'-' means the log doesn’t need to be synchronized with the storage. Meaning that a log message will not appear right after you’ve sent the log.

These lines define the facilities and the levels of the logs.



Page 14

Ways to log…• The simplest way…use the command

"logger"… I’m choosing:- the “user” facility with…- the “debug” level.

[tywong@linux] $ logger -p user.debug "hi hi"[tywong@linux] $ _

auth,authpriv.* /var/log/auth.log*.*;auth,authpriv.none -/var/log/syslog#cron.* /var/log/cron.logdaemon.* -/var/log/daemon.logkern.* -/var/log/kern.loglpr.* -/var/log/lpr.logmail.* -/var/log/mail.loguser.* -/var/log/user.log

[tywong@linux] $ logger -p user.debug "hi hi"[tywong@linux] $ cat /var/log/user.log......Mar 26 01:03:06 pc91087 tywong: hi hi[tywong@linux] $ _



Page 15

The facilities…

Some Facilities Descriptions

LOG_AUTH From authorization programs: login, sudo, etc…

LOG_KERN Generated by the kernel.

LOG_LOCAL[0-7] Reserved by local use.

LOG_SYSLOG From the syslogd itself.

LOG_USER From user processes (default)

Very interesting that segmentation faults are logged by kernel!

[tywong@linux] $ logger "hi hi"

This, by default, goes to LOG_USER facility.



Page 16

The levels…

All Levels Descriptions

LOG_EMERG Emergency (system is unusable)

LOG_ALERT Condition that must be fixed immediately

LOG_CRIT Critical (e.g., hardware error)

LOG_ERR Error

LOG_WARNING Warning

LOG_NOTICE Normal, but worth noticing issues

LOG_INFO Informational

LOG_DEBUG Debug message

The level is increasing.

That means the importance of the message.

This is for the administrator or the programmer to look for their logs of interest.



Page 17

Back to the configuration…• Knowing that there are different facilities and

levels…auth,authpriv.* /var/log/auth.log*.*;auth,authpriv.none -/var/log/syslog#cron.* /var/log/cron.logdaemon.* -/var/log/daemon.logkern.* -/var/log/kern.loglpr.* -/var/log/lpr.logmail.* -/var/log/mail.loguser.* -/var/log/user.log

local0.*;local0.!=debug -/var/log/csc4140.loglocal0.debug -/var/log/csc4140_debug.log

The rules together say:- for debug level, go to “csc4140_debug.log”- otherwise, go to “csc4140.log”.



Page 18

Using the new facility…

[root@linux] # /etc/init.d/sysklogd restart[root@linux] # sudo su - tywong[tywong@linux] $ logger -p local0.debug "debugging"[tywong@linux] $ logger -p local0.alert "alert you"[tywong@linux] $ cat /var/log/csc4140.logMar 26 02:50:06 pc91087 tywong: "alert you"[tywong@linux] $ cat /var/log/csc4140_debug.logMar 26 02:50:06 pc91087 tywong: "debugging"[tywong@linux] $ _

Let’s try this out!



Page 19

Logging inside programs…

#include <stdio.h>#include <syslog.h>

int main(void) {openlog("simple_log", LOG_PID, LOG_LOCAL0);

syslog(LOG_DEBUG, "%s\n", "hello world");

closelog();return 0;

}

Mar 26 03:00:00 pc91087 simple_log[1410]: "hello world"

You can freely change this

string.

This sets the PID to be displayed.

This logs to the debug level.



Page 20

Logging over I/O Redirection

Pros Write less, gain more.

The logging service is already there, why not using it.

Program changes.

In order to use the logging, every fprintf() or printf() statement has to be converted into syslog()statement.

Cons

Non-interoperability

In case that the code is ported to Windows, I/O redirection is a more compatible choice.



Page 21

Conclusion…Server Coding Rules

(1) fork() and let the parent dies.This creates a “background process”.

(2) Close all the unwanted files.It is too dangerous to leave this time bomb unattended.

(2) Before exec(), switch the process group of the server process.This avoids unexpected termination from broadcast signals.

(3) Implement the PID file technique.This avoids the executions of multiple instances of the same program.

(4) Implement graceful termination.This allows your program to be terminated in a consistent state.

(5) Log as much as possible.A server is supposed to be running without a terminal.You are recommended to redirect the standard output and error streams.Use syslog() is also a good choice.

Documents

What Do u Mean by This Define Log_local 5 (21