리눅스 드라이버 실습 #3

리눅스 드라이버 실습 #32015/Jun/15

박상호

Copyright © 2015 S-Core Co., Ltd. All rights reserved | Con-fidential

I. #2 Wrap-up

II. IRQ(Interrupt Request)

III. poll(2)

IV. Test

실습 내용


실습 #2 Wrap-up

- 4 -Copyright © 2015 S-Core Co., Ltd. All rights reserved | Con-fidential

Time sharing

프로그램 A -> 프로그램 B -> 프로그램 C

CPU Execution

Context Switch: schedule() in kernel

Save registers

Load registers

continue to execute

A B C A B CTime


Simple example

test.c

$ cat test.c#include <stdio.h>

int i = 0;intmain(){ i++; i+=2; i+=3; printf("i=%d\n", i); return 0;}$ gcc –g test.c$ objdump –S a.out…$ objdump –S a.out | less

#include <stdio.h>

int i = 0;intmain(){ 40052d: 55 push %rbp 40052e: 48 89 e5 mov %rsp,%rbp i++; 400531: 8b 05 0d 0b 20 00 mov 0x200b0d(%rip),%eax # 601044 400537: 83 c0 01 add $0x1,%eax 40053a: 89 05 04 0b 20 00 mov %eax,0x200b04(%rip) # 601044 i+=2; 400540: 8b 05 fe 0a 20 00 mov 0x200afe(%rip),%eax # 601044 400546: 83 c0 02 add $0x2,%eax 400549: 89 05 f5 0a 20 00 mov %eax,0x200af5(%rip) # 601044 i+=3; 40054f: 8b 05 ef 0a 20 00 mov 0x200aef(%rip),%eax # 601044 400555: 83 c0 03 add $0x3,%eax 400558: 89 05 e6 0a 20 00 mov %eax,0x200ae6(%rip) # 601044 printf("i=%d\n", i); 40055e: 8b 05 e0 0a 20 00 mov 0x200ae0(%rip),%eax # 601044 400564: 89 c6 mov %eax,%esi


Wait & wake

wait_for_completion

complete

void wait_for_completion(struct completion *);void wait_for_completion_interruptible(struct completion *);void wait_for_completion_killable(struct completion *);void wait_for_completion_timeout(struct completion *, unsigned long timeout);void wait_for_completion_interruptible_timeout(struct completion *, unsigned long timeout);void wait_for_completion_killable_timeout(struct completion *, unsigned long timeout);bool try_wait_for_completion(struct completion *x);

void complete(struct completion *);void complete_all(struct completion *);


Wait(1)

wait

Interruptible?

Signal 을 받을 때 , 깨어남

Check the return value: -ERESTARTSYS

Uninterruptible (By default)

Signal 을 받아도 깨어나지 않음

$ kill -l 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR111) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO 30) SIGPWR31) SIGSYS 34) SIGRTMIN 35) SIGRTMIN+1 36) SIGRTMIN+2 37) SIGRTMIN+338) SIGRTMIN+4 39) SIGRTMIN+5 40) SIGRTMIN+6 41) SIGRTMIN+7 42) SIGRTMIN+843) SIGRTMIN+9 44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+1348) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-1253) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9 56) SIGRTMAX-8 57) SIGRTMAX-758) SIGRTMAX-6 59) SIGRTMAX-5 60) SIGRTMAX-4 61) SIGRTMAX-3 62) SIGRTMAX-263) SIGRTMAX-1 64) SIGRTMAX$ man 7 signal


wait_queue_t

wait_queue_t

struct __wait_queue_head { spinlock_t lock; struct list_head task_list;};typedef struct __wait_queue_head wait_queue_head_t;


Critical Section

2 개 이상의 Context 에서 동시에 메모리 접근할 경우 , 보호되어야할 구간

Echo driver 에서도 read/write 이 같은 데이터를 동시에 접근하고 변경하고 있음

$ cat test.c#include <stdio.h>

int i = 0;intmain(){ i++; i+=2; i+=3; printf("i=%d\n", i); return 0;}$ gcc –g test.c$ objdump –S a.out…$ objdump –S a.out | less


Synchronization

Synchronization: critical section 을 보호하기 위한 방법

Kernel 에서 제공되는 Synchronization primitives

Low-level: Memory barrier, Atomic operations, Synchronize with interrupts, Spin locks

High-level: Completion, Mutex, Semaphore, read-write lock

Spin lock

Spinning: 계속 메모리의 value 를 체크

장점 : lock 을 잡는데 소요되는 시간이 제일 짧음

단점 : CPU 를 계속 사용하게 됨 . Critical section 이 긴 경우 , 사용하면 안됨

Mutex

Sleep-based synchronization

장점 : Lock 을 잡지 못하면 , CPU 를 계속 사용하지 않음

단점 : Lock 을 잡지 못하고 sleep 하게 되는 overhead


IRQ(Interrupt Request)


Polling & Interrupt

장치가 네트워크 패킷을 전송 완료하였는지 어떻게 체크할지 ?

또는 장치의 상태 변화를 어떻게 체크할지 ?

Polling & Interrupt

Polling: CPU 에서 계속 / 주기적으로 장치 상태 변화 / 완료 여부를 체크

Interrupt: 상태 변화 / 완료시 장치에서 CPU 에게 interrupt 를 보냄

IRQ(Interrupt Request)H/W 가 processor 에게 현재 실행하고 있는 프로그램을 멈추고 , 별도의 프로그램 (Interrupt handler) 를 실행하도록 요청

$ cat /proc/interruptsIRQ Line 0 … 15

Interrupt

ProgrammableInterruptController

http://en.wikipedia.org/wiki/Interrupt_request_(PC_architecture)


IDT(Interrupt Descriptor Table)


Exceptions

x86 exceptions and interrupts

Vec-tor

Description Signal

0 Division by zero SIGFPE

1 Debug SIGTRAP

2 NMI(Non-maskable external inter-rupt)

-

3 Breakpoint (INT 3 instruction) SIGTRAP

4 Overflow SIGSEGV

5 Bound range exceeded SIGSEGV

6 Invalid instruction SIGILL

7 No coprocessor SIGSEGV

8 Double fault SIGSEGV

9 Coprocessor segment overrun SIGFPE

10 Invalid TSS SIGSEGV

11 Segment Not Present SIGBUS

12 Stack Segment Fault SIGBUS

13 General Protection SIGSEGV

14 Page fault SIGSEGV

15 Spurious Interrupt -

16 Floating-Point Error SIGFPE

17 Alignment Check SIGSEGV

18 Machine check -

19 SIMD Floating-Point Exception SIGFPE

32-255

Available for external input -

IRQ # Vector Device

0 32 Timer

1 33 Keyboard

2 34 PIC cascading

6 38 Floppy

14 46 Disk controller

- 128(0x80) System call (linux)

- 129-238 External inputs

- 239 Local APIC timer interrupt

- 251-253 Interprocessor interrupts

arch/x86/include/asm/traps.h


Polling & Interrupt

Interrupt handler execution


Polling & Interrupt

Polling & Interrupt

대부분의 드라이버는 interrupt 방식임 .

일부 네트워크 드라이버에서 polling 방식 차용

Polling Interrupt

방식 Driver 가 계속 상태를 체크 Device 가 CPU 에 변화를 알려줌

장점 빠른 처리 (Context Switch 가 없음 ) 효율적인 CPU 사용(CPU 가 다른 일을 할 수 있음 )

단점 비효율적인 CPU 소모 (Busy waiting) Interrupt Context 로 전환 비용


Interrupt Context

Device Driver

User-driven request handling -> Normal context

file_operations

open

read

write

release

poll

ioctl

ISR: Interrupt Service Routine (Interrupt handler) -> Interrupt context

즉 , interrupt-handler 와 normal context 사이에도 race condition 이 발생할 수 있음

두 context 사이의 race condition 을 방지하기 위해서는 spin_lock_irqsave 를 사용하여야 함 .

interrupt handler 에서 sleep 하면– interrupt disable 상태로 handler 가 sleep 하면 시스템이 응답 없음

Spin_lock 사용시 - normal context 에서 spin_lock 을 잡은 상태에서 interrupt handler 가 수행하게 되면 , 계속 spinning 하게 되어 역시 시스템이 응답 없음

따라서 , spin_lock_irq/spin_lock_irqsave 을 사용해야함 – spin_lock 을 잡으며 , interrupt disable 하며 , irq 상태를 리턴


poll(2)


Reader/Writer

fd = open(argv[1], O_RDONLY);…while ret = read(fd, buf, sizeof(buf)); … ret = write(STDOUT_FILENO, buf, ret)); …close(fd);

readerfd = open(argv[1], O_WRONLY|O_CREAT|O_TRUNC);…while ret = read(STDIN_FILENO, buf, sizeof(buf)); … ret = write(fd, buf + i, len - i)); …close(fd);

writer

Wait on read(/dev/echo)

write(stdout)

Wait on read(stdin)

write(/dev/echo)


rw: reader & writer

Reader & Writer

Stdin 과 /dev/echo 두 곳의 입력을 동시에 기다려야 함

read(/dev/echo) 시에 stdin 입력이 들어올 경우 ? 그 반대는 ?

동시에 입출력을 기다려서 처리할 수 있는 프로그래밍 방법 필요 -> poll(2) / select(2) / epoll(2)


write(stdout)

Wait on read(stdin)

write(/dev/echo)


poll(2)

poll(2)

NAME poll, ppoll - wait for some event on a file descriptor

SYNOPSIS #include <poll.h>

int poll(struct pollfd *fds, nfds_t nfds, int timeout);

DESCRIPTION poll() performs a similar task to select(2): it waits for one of a set of file descriptors to become ready to perform I/O. The set of file descriptors to be monitored is specified in the fds argument, which is an array of structures of the following form: struct pollfd { int fd; /* file descriptor */ short events; /* requested events */ short revents; /* returned events */ }; The caller should specify the number of items in the fds array in nfds. The timeout argument specifies the minimum number of milliseconds that poll() will block. … Specifying a negative value in timeout means an infinite timeout. Specifying a timeout of zero causes poll() to return immediately, even if no file descriptors are ready.

On success, a positive number is returned; this is the number of structures which have nonzero revents fields (in other words, those descriptors with events or errors reported). A value of 0 indicates that the call timed out and no file descriptors were ready. On error, -1 is returned, and errno is set appropriately.


poll(2)

poll(2)

The bits that may be set/returned in events and revents are defined in <poll.h>:

POLLIN There is data to read.

POLLPRI There is urgent data to read (e.g., out-of-band data on TCP socket; pseudoterminal mas-ter in packet mode has seen state change in slave).

POLLOUT Writing now will not block.

POLLRDHUP (since Linux 2.6.17)Stream socket peer closed connection, or shut down writing half of connection. The _GNU_SOURCE feature test macro must be defined (before including any header files) in order to obtain this definition.

POLLERR Error condition (output only).

POLLHUP Hang up (output only).

POLLNVAL Invalid request: fd not open (output only).

(POLLRDNORM Equivalent to POLLIN.)

(POLLRDBAND Priority band data can be read (generally unused on Linux).)

(POLLWRNORM Equivalent to POLLOUT.)

(POLLWRBAND Priority data may be written.)


Step 1: reader w/ poll(2)

rw.c based on reader.c

… struct pollfd fds[1]; char buf1[128];… buf1len = sprintf(buf1, “%d:”, getpid()); /* process id */

fd = open(argv[1], O_RDWR);…

memset(fds, 0, sizeof(fds)); fds[0].fd = STDIN_FILENO; fds[0].events = POLLIN;… while(1) { ret = poll(fds, 1, -1); if (ret < 0) { fprintf(stderr, “Failed to poll: %s\n”, strerror(errno)); ret = -errno; goto end; }

if (fds[0].revents & POLLIN) { /* read & write */ /* write w/ pid = buf1 */ }


Step 2: rw

writer.c 로 확장


write(stdout)

Wait on read(stdin)

write(/dev/echo)

<PID>:ECHO:


Driver 에서 poll 지원


Virtual Filesytem Switch

reader/writer

VFS(Virtual File Switch): 디바이스 드라이버 , 파일 시스템의 file_operations 호출 fd -> fd 테이블의 index -> struct file * -> file_operations 호출

echo driver

open(“/dev/echo”, O_RDONLY)

__open(inode, file)

read(fd, buf, sizeof(buf))

__read(file, buf, len, off)

close(fd)

__release_(inode, file)


poll in file_operations

1538 struct file_operations {1539 struct module *owner;1540 loff_t (*llseek) (struct file *, loff_t, int);1541 ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);1542 ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);1543 ssize_t (*aio_read) (struct kiocb *, const struct iovec *, unsigned long, loff_t);1544 ssize_t (*aio_write) (struct kiocb *, const struct iovec *, unsigned long, loff_t);1545 ssize_t (*read_iter) (struct kiocb *, struct iov_iter *);1546 ssize_t (*write_iter) (struct kiocb *, struct iov_iter *);1547 int (*iterate) (struct file *, struct dir_context *);1548 unsigned int (*poll) (struct file *, struct poll_table_struct *);1549 long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);1550 long (*compat_ioctl) (struct file *, unsigned int, unsigned long);1551 int (*mmap) (struct file *, struct vm_area_struct *);1552 int (*mremap)(struct file *, struct vm_area_struct *);1553 int (*open) (struct inode *, struct file *);1554 int (*flush) (struct file *, fl_owner_t id);1555 int (*release) (struct inode *, struct file *);1556 int (*fsync) (struct file *, loff_t, loff_t, int datasync);1557 int (*aio_fsync) (struct kiocb *, int datasync);1558 int (*fasync) (int, struct file *, int);1559 int (*lock) (struct file *, int, struct file_lock *);1560 ssize_t (*sendpage) (struct file *, struct page *, int, size_t, loff_t *, int);1561 unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, un-signed long, unsigned long);…

http://lxr.free-electrons.com/source/include/linux/fs.h#L1538




poll(2)

VFS 에서 do_sys_poll 의 역할

User 영역에서 struct pollfd array 를 kernel 영역으로 복사

루프

fd 마다 file_operations 의 poll 이 있는지 체크하여 호출

없을 경우는 DEFAULT_MASK(POLLIN | POLLOUT | POLLRDNORM | POLLWRNORM) & event 리턴

각 poll 함수에서는

waitqueue 에 추가 : 이벤트가 발생했을 때 , wake_up 이 가능하도록

이벤트 여부 검사 & 리턴

이벤트가 발생했다면 , waitqueue cleanup 하고 리턴

이벤트 발생하지 않으면 , sleep


poll of echo driver

…#include <linux/poll.h> /* poll */#include <linux/sched.h> /* wake_up_interruptible */…

struct echo_device {… wait_queue_head_t waitqueue; /* waitqueue for poll */…};…static struct file_operations fops ={… .poll = __poll,};… init_waitqueue_head(&_dev.waitqueue); // at initialization… wake_up_interruptible(&_dev.waitqueue); // at write…


poll of echo driver

// called when 'poll' system call is calledstatic unsigned int __poll(struct file *file, poll_table *wait){ unsigned int ret;

poll_wait(file, &_dev.waitqueue, wait);

ret = 0; mutex_lock(&_dev.lock); /* is there data to read */ if (_dev.buf[_dev.pos] != '\0') { ret |= POLLIN | POLLRDNORM; } mutex_unlock(&_dev.lock); ret |= POLLOUT | POLLWRNORM;

return ret;}


Execute

$ sudo insmod echo$ sudo mknod /dev/echo c 250 0$ sudo chmod 777 /dev/echo$ ../reader_writer/rw /dev/echo1234ECHO:2938:1234asdfECHO:2939:asdf


테스트

Software

리눅스 드라이버 실습 #3