Ubuntu (operating system)UBUNTU Operating system concepts and its implementation

Submitted to:

Sir Amir Raza

Submitted by:

Noman Iqbal

MIT-10-02

3rd SEMESTER

Department Of Information Technology

Bahauddin Zakariya University

Multan.

Introduction

General Information.

Ubuntu is a computer operating system based on the Debian Linux distribution and distributed as free and open source software. It is named after the Southern African philosophy of Ubuntu ("humanity towards others").Ubuntu is designed primarily for use on personal computers, although a server edition also exists.

Ubuntu is sponsored by the UK-based company Canonical Ltd., owned by South African entrepreneur Mark Shuttleworth. Canonical generates revenue by selling technical support and services related to Ubuntu, while the operating system itself is entirely free of charge. The Ubuntu project is committed to the principles of free software development; people are encouraged to use free software, improve it, and pass it on.

Ubuntu 11.04 - General Overview

General Information.

UBUNTU Linux 11.04 (with code name natty narwhal) is a GNU/LINUX stable

distribution provided by Canonical Ltd. and released on April 2011. It supports Intel x86

(IBM-compatible PC), AMD64 (Hammer) and PowerPC (Apple iBook and Powerbook,

G4 and G5) architectures. It is available in both 32-bit and 64-bit versions. It is based on

2.6.38.2. kernel release and comes with UNITY graphical user interface.

Features against competition - Target group.UBUNTU 11.04 Desktop edition, is a free Operating System committed to  the

principles of open source software development. It is and will always be free of charge,

including updates. The release will be supported for 18 months. But this is not all.

Through “UBUNTU software Center”, it provides a database for easy and fast access to

free, compatible, open source applications that fit almost all needs.

Useful everyday commands and symbols

Comman Description

ls List files and folders. Example: ls -l. Shows all files in folder, including hidden ones.

cd Change folder. Example: 'cd ~/Documents' will move to Documents folder in the /home/”current user” folder

cp Copy file or folder. Example: cp file.doc Desktop/, will copy file.doc from current path to /home/”current user”/desktop

mv Move files or folders. If no destination path is provided, renames current file/folder

rm Deletes files

nano and vi Non graphical editors, Gedit equievelant

man View the manual for a specified command

locate Finds a specified file. It used a database which can be updated manually using ”sudo updatedb”

rmdir Deletes folder

mkdir Creates folder

wget Powerful download program

apt-get (Advanced Packaging Tool)

> Redirects the output of a command to a specified location/file

| Pipe. Uses the output of one command as input for another

Ubuntu 11.04 - Process management

Introduction

UBUNUT Linux11.04 is based on 2.6.38 Linux kernel. As far as the process management is concerned, UBUNTU Linux11.04 has preemptive kernel and supports Symmetric Multiprocessing (SMP).

There are two spaces that we identify in process circle. The kernel space and the

Processes and threads.

Processes and threads are not that different for linux. Both fork() and exec() are

used to create processes. Threads are created by the clone() system call. Clone()

creates a child process allowing it to use shared information with the parent process by

sets of flags that are passed to it. This how a thead is described in other Operating

Systems. Exec() is the system call used to run new program whereas fork() is the

system call to create a new process. In general it can be said that fork() is similar to

clone() when no flags are passed to the new process. (Operating Systems Concepts, p.

815)

Process properties

Process poperties fall into three groups:

1. Process identity

Process ID (PID)

It is the unique identifier of the process. It is used to specify the process to the

OS. It does not change until the process termination.

Credentials

Also known as User ID (UID) and Group ID (GID). It determines the rights and

privilegdes of the process to access system resources.

Personality

It can modify the semantics of certain system calls. It is used primarly by

emulation libraries in order to request the combatible system call with certain varieties of

UNIX.

2. Process environment

It “is inherited from its aprent process and is composed of two null-terminated vectors;

the argument vector and the environment vector . The environment vector is a list of

“NAME=VALUE” paris that associate named enviroment variables with arbitary textual

values. eg. The LANG variable determines the language that is to be used to display

system messages.

2. Process Context

It represents the state of the process and it constantly changes. It inculdes the

parts listed in the table 1 below.

Description

1 Scheduling Context

2 Accoutning

3 File table

4 File-sysstem context

5 Signal-handler table

6 Virtual memory context

Table 1. Parts of the Process context (Operating System Concepts, p.814)

Scheduling

UBUNTU Linux 11.04 have two seperate process-scheduling algorithms. “One is

time-sharing for fair, preemptive scheduling among multiple processes; the other is

designed fo real-time tasks.” “Real-time algorithms are also supported, for special time-

critical applications that need precise control over the way in which runnable processes

are selected for execution.”

The time sharing algorithms are three.

SCHED_OTHER

SCHED_BATCH

SCHED_IDLE

For the time sharing algorithms, the standard Round-Robin method is used. It is

a First-Comes-First-Served (FCFS) method, but also assingning a time quanta for each

process, creating a loop ending with the init() system call, which is the parent of all

processes.

Two seperate priority ranges are used. A real time range from 0-99 and a nice

value ranging from 100 to 140. In this scheme the numerically lower processes have

higher priority. When a task is runnable, it enters a runqueue, mainted by the kernel.

The runqueue includes two arrays. The active list, which contains the processes to be

executed, and the expired list, which contains the tasks that have expired. If the time

quanta does not allow the task to be completed, then it is considerd as expired and it

goes back to the expired array list of the runqueue. When the acitve array of the active

list = 0 then the expired list becomes the active one and vice versa. The runqueue is

processor indepented.

The nice priority value is determined by how much time the task has been

sleeping. It is dynamically changed and the scheduler infavors interactive tasks, which

are tasks that have longer sleep times. The nice priority value is recalculated when the

task enters the expired list.

The real time algorithms are the FCFS and the Round-Robin, which was

described above, with the addition of a priority to the one of its scheduling class. When

two processes have the same class, the one waiting in queue the lognest has higher

priority. For processesof different class the one with the hight class has the higher

priority.

Administration utilities used from the command-line

There are several commands that be used to monitor and manage the running

processes of the system

In order to interact with a task in terminal, we need to know either the PID or the

name of the task. The table below summarises the most useful commands for process

management through the terminal. The description of each command as well as the

available arguments can be viewed by typing (command) --help, or man (the name of

the command)

Name Description

1 ps With the appropriate arguments it can show all the running processes, the process runnning from a user

2 pstree Shows the runnning processes in a tree scheme

3 kill Kill a process by PID

4 killall Kill a process by name

5 nice Sets the priority of new processe

6 renice Changes the priority of a currently running process

7 skill Reports process status

However, I believe that the most usefull command for the process management

throught the terminal is the command top. It is a real time non graphical “System

Monitor”.

Althought the number of the taks running is bigger and because it changes

dynamically, the best proposed use of it is to use the results of the top command

exporting them in home folder in a text file, like this

top >> ~/top.txt, which will overwrite a text file in the home folder with the name

top.

It will provide with all the necessary information to manipulate the running tasks

in the system. The “h” key shows the help menu na the “q” is used to exit the top

command.

Ubuntu 11.04 - Memory management

Overview – Allocation

UBUNTU as a GNU/Linux distribution is designed for different architectures. Due

to this characteristic, UBUNTU separates physical memory into three different zones:

zone_DMA

zone_NORMAL

zone_HIGHMEM

Zone_DMA is the zone that includes the first 16MB of physical memory. It is

consisted of the DMA-able pages (Direct Access Memory). This happens because

certain hardware, Industry Standard Architecture (ISA) devices, can only access the

lower 16MB of physical memory.

Zone_NORMAL is the zone that includes the normally addressable pages up to

the first 896 MB of physical memory. The remaining physical memory (>896 MB) is

allocated from zone_HIGHMEM. It is referred as high memory and includes the

dynamically mapped pages.

“The primary physical-memory manager in the Linux kernel is the page allocator”.

There is one allocator for each zone, responsible for allocating, freeing all physical

pages for each zone, and capable of allocating ranges of physically contiguous pages

on request, but even though some allocations may require a page from a particular

zone, a normal allocation can come form zone_DMA or zone_NORMAL. However, it is

preferable to use a page from the particular zone, mostly to save the DMA space, but if

needed and memory is low, Kernel can use whichever zone is available and suitable.

(Robert Love, Linux Kernel Development Second Edition). It is important to note that

many kernel operations can only take place using zone that utilises available physical

memory, reducing the percentage of internal fragmentation. However, as of 2.6.38

kernel, the one UBUNTU 11.04 uses the transparent huge page (THP) feature is added,

which takes advantage of the page sizes that contemporary processors can handle

simultaneously

UBUNTU do very high use of virtual memory. mmap() service is responsible for

mapping files or devices into memory. “Linux reserves for its own internal use a

constant, architecture-dependent region of the virtual memory address space of every

process. The page-table entries that map to these kernel pages are marked as

protected” Two regions are used, one where the core of kernel, along with all pages

allocated by the normal page allocator reside, and another which is reserved for no

specific purpose. The vmalloc() service is used to allocate an arbitrary number of

physical pages of memory that may not be contuguous into a single region of virtually

contiguous kernel memory.

The mmap() service has flag arguments that transform the mapping to shared

with other processes, or private creating a private copy-on0write mapping. The virtual

mapping procedure is directly connected to the way the process was created. If it is

created through the exec() servicem then a new mapping address is provided. If it is

created through fork() then the parent's pages are directly copied to the child's, resulting

that the parent and child share physical pages in memory. UBUNTU 11.04 use the least

frequently used (LFU) as page out policy .It is through the age variable of a page that it

is chosen which page to swap.

Memory Management OS Utilities

The only graphical tool about memory management that comes with UBUNTU

installation is the System Monitor. One can reach it from System Settings → System

Monitor. As it can be seen, the only available information one can get is the amount and

percentage of memory and swap currently being used. A graphical representation over

time is created helping us to visualize the constant chagnes in the values. However,

these information are poor for an advanced user and even less for the system root user.

More detailed utilities are available through the command-line, most of which are only

available for the root group.

PID : Process ID USER: User the process belongs to

PR: Priority

NI: Nice value

VIRT: Virtual Image

RES: Resident size

SHR:Shared memory size

%CPU: CPU usage

%MEM: Memory usage

TIME+:CPU Time

COMMAND: command name/line

Additional commands and files containing memory information are shown below:

free -m Limited memory information

/proc/meminfo Full memory information

/proc/interrupts System interrups served per CPU

uptime System uptime

vmstat -s Virtual memory information

sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches"

Drops caches

pagesize Page size currently used

uname -r Kernel version (PAE included)

sudo swapoff -a Turning swap off

sudo swapon -a Turning swap on

Ubuntu 11.04 - File Systems

File systems and UBUNTU 11.04

Ubuntu 11.04 support the Linux Logical Volume Manager, which allows us to increase

the size of a partition online, while the system is running.

In addition, Ubuntu 11.04 allows us to use different devices as mount points for several

“heavy/crucial” directories. These are:

/boot, the directory containing the basic instructions to load the OS

/ (root) directory

/home, the user(s) home directory, equivalent to “My documents” folder in MS Windows

A full list of the supported file systems in UBUNTU 11.04 can be found using the “cat”

command in the virtual file system /proc as shown below.

cat /proc/filesystems

Table 1. Show some supported file systems in UBUNTU 11.04 according to Ubuntu

minix is the file system used in the Minix operating system, the first to run under Linux. It has a number of shortcomings: a 64MB partition size limit, short filenames, a single timestamp, etc. It remains useful for floppies and RAMdisks.

ext an elaborate extension of the minix file system. It has been completely superseded by the second version of the extended file system (ext2) and has been removed from the kernel (in 2.1.21).

ext2 the high performance disk file system used by Linux for fixed disks as well as removable media. The second extended file system was designed as an extension of the extended file system (ext). ext2 offers the best performance (in terms of speed and CPU usage) of the file systems supported under Linux.

msdos the file system used by DOS, Windows, and some OS/2 computers. msdos filenames can be no longer than 8 characters, followed by an optional period and 3 character extension.

umsdos an extended DOS file system used by Linux. It adds capability for long filenames, UID/GID, POSIX permissions, and special files (devices, named pipes, etc.) under the DOS file system, without sacrificing compatibility with DOS.

vfat an extended DOS file system used by Microsoft Windows95 and Windows NT. VFAT adds the capability to use long filenames under the MSDOS file system.

ntfs replaces Microsoft Window's FAT file systems (VFAT, FAT32). It has reliability, performance, and space-utilization enhancements plus features like ACLs, journaling, encryption, and so on.

nfs the network file system used to access disks located on remote computers.

However, support for additional file systems can be added by adding additional modules

to kernel.

As it can be seen above, UBUNTU 11.04 supports several file systems. The result when

using the cat /proc/filesystems – command, will include several virtual file systems connected to

no devices, that are used from Ubuntu kernel for administrative actions, such as memory

caching. e.g.

“ramfs, a very simple filesystem that exports Linux's disk caching mechanisms (the page

cache and dentry cache) as a dynamically resizable RAM-based filesystem.

rootfs, a special instance of ramfs (or tmpfs, if that's enabled), which is

always present in 2.6 systems. You can't unmount rootfs for approximately the

same reason you can't kill the init process; rather than having special code

to check for and handle an empty list, it's smaller and simpler for the kernel

to just make sure certain lists can't become empty.”

Adding to those above, the swap file system must be noted as well, even though it is a

raw file system, used for memory caching. Swap is a very important parameter in Linux.

The ones available to use in a physical drive during a clean installation, using the default

partitioning tool are:

ext2

ext3

ext4

reiserfs

xfs

jfs

btrfs

A very brief comparison of the File systems most commonly found in UBUNTU.

File System Max File Size Max Partition Size

Journaling Notes

Fat16 2 GB 2 GB No Legacy

Fat32 4 GB 8 TB No Legacy

NTFS 2 TB 256 TB Yes For Windows Compatibility) NTFS-3g is installed by default in Ubuntu, allowing Read/Write support)

ext2 2 TB 32 TB No Legacy

ext3 2 TB 31 TB Yes Standard linux filesystem for many years. Best choice for super-standard installation.

ext4 16 TB 1EB Yes Modern iteration of ext3. Best choice for new installations where super-standard isn't necessary.

reiserfs 8 TB 16 TB Yes No longer well-maintained.

JFS 4 PB 32 PB Yes (metadata) Created by IBM - Not well maintained.

XFS 8 EB 8 EB Yes (metadata) Created by SGI. Best choice for a mix of stability and advanced journaling.

(https://help.ubuntu.com/community/LinuxFilesystemsExplained)

Tools for the ext2,ext3,ext4 and XFS file systems in Ubuntu.

The available tools for file system information and modifications, are separated ion this

paper in two parts. The one where the only the graphical tools are explained, and the second

one where the commands in terminal are briefly explained.

GUI tools

A fast overview of the available file systems can be found in the system monitor, in the file

systems tab.

Further actions and information for the available storage devices can be found in the

Disk Utility in the Control Center.

Disk Utility provide us with an overview of the storage devices currently connected in the

system. It also provides us with an easy graphical interface to:

get hardware information

mount devices

unmount devices

check the filesystem in the device

edit partitions

delete partitions

format volumes

Graphical tools for fie systems, as for any other administrative actions, are poorer

comparing to the equivalents in terminal. They addressed to normal users and available actions

are limited, e.g define block size, cannot be processed thought them.

Terminal Commands

They are addressed to advanced users and the can modify any available option for a file

system. This is why they must be used as superuser, or by borrowing su privileges. Additional

information, about parameters and syntax, are provided in the links below.

blkid, used to find the Universally Unique Identifier (UUID) for a device, which is prefered

from Ubuntu over the deveice path, eg /dev/sda, since it can change during boot.

mkfs, used to create a file system

fsck, used to check and repair file system

fdisk

df, used to report file system usage

badblocks, used to check a device for bad blocks

umount and mount, used to unmount and mount devices

Ubuntu 11.04 - Device management

Introduction

UBUNTU has support for all types of hardware built into the kernel. (Ubuntu pocket

guide and reference, p.19). To the extent possible, all device drivers appear as normal files.

Devices are separated in three classes:

block devices

Here belong all the devices that allow random access to completely independent, ficed-size

blocks of data, such as HDDs, floppy disks, CD-ROMs, flash memory. They are accessed

randomly.

character devices

In this category belong most other devices, most of which are pointing devices such as

keyboards, mice etc. They are accessed serially.

network devices

Here we find the network devices that users cannot communicate directly. Their transfer data by

opening connection with the kernel's network subsystem.

A fundamental difference in Ubuntu 11.04, based on their kernel version, 2.6.38 is the

use of deadline I/O shceduler in order address the starvation issue. The deadline for read

request is 0,5 second and for write requests is 5 seconds. It maintains a sorted queue, a read

and a write queue.

Four are the key folders for devices in UBUNTU, show in the table below:

Folder Description

/dev Virtual files representing hardware devices

/media Contains folder where storage devices can be mounted

/proc Virtual folder containing file representing stats and settings

/etc System (global) configuration files

UBUNTU does not have a “Computer” view like windows. Storage devices are mounted

in folders inside the /media folder. Moreover, for HDDs Master Boot record is important,

because bootloader (GRUB 2.0) cannot be after the 1024th cyclinder, since it may cause

system to halt at boot.

Through kernel updates the drivers are updated and more are included. However, it is

possible to compile drivers in kernel. However, it is addressed to advanced users only. The

administration of the devices using graphical tools is limited, compared to the tools available in

terminal. The utilities can be found in the System settings → “Hardware” and “System” section.

Additionally the gnome-device-manager package can be installed through the synaptic

manager or through terminal,

$sudo apt-get install gnome-device-manager

in order to graphically list all the available connected devices. “Connected” devices does

not mean “operational” too.

System administrator must count in terminal to list and compare values from the /proc

folder and /dev for the problematic device, so as to verify the amount of interrupts, mapping,

memory allocation addresses, pages etc to figure out where the problem is located.