D4.1 – API Speciﬁcationapi.ning.com/files/8CzpakKglFRj9JYZrBB2sNfPkGGP-6D7U0xvhIpeaGaZQpn... · This deliverable describes the APIs that can be used to extend the existing functionality

Project Number 611411

D4.1 – API Specification

Version 1.06 November 2014

Final

EC Distribution

aicas and University of York

Project Partners: aicas, Bosch, CNRS, Rheon Media, The Open Group, University of Stuttgart, University of York

Every effort has been made to ensure that all statements and information contained herein are accurate, howeverthe DreamCloud Project Partners accept no liability for any error or omission in the same.

© 2014 Copyright in this document remains vested in the DreamCloud Project Partners.


Project Partner Contact Information

aicas BoschFridtjof Siebert Jochen HärdtleinHaid-und-Neue Strasse 18 Robert-Bosch-Strasse 276131 Karlsruhe 71701 SchwieberdingenGermany GermanyTel: +49 721 66396823 Tel: +49 711 811 24517E-mail: [email protected] E-mail: [email protected]

CNRS Rheon MediaGilles Sassatelli Raj PatelRue Ada 161 20 Leighton Avenue34392 Montpellier Pinner Middlesex HA5 3BWFrance United KingdoTel: +33 4 674 18690 Tel: +44 7547 162920E-mail: [email protected] E-mail: [email protected]

The Open Group University of StuttgartScott Hansen Bastian KollerAvenue du Parc de Woluwe 56 Nobelstrasse 191160 Brussels 70569 StuttgartBelgium GermanyTel: +32 2 675 1136 Tel: +49 711 68565891E-mail: [email protected] E-mail: [email protected]

University of YorkLeandro IndrusiakDeramore LaneYork YO10 5GHUnited KingdomTel: +44 1904 325 570E-mail: [email protected]

Page ii Version 1.0Confidentiality: EC Distribution

6 November 2014


Contents

1 Executive Summary 1

2 Background 2

3 Requirements 4

4 Low-Level Resource Control 7

4.1 Realtime Java and OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4.1.1 Base Java environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4.1.2 Base OS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4.2 Controlling CPU frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4.3 Performance Optimization and Compilation . . . . . . . . . . . . . . . . . . . . . . 8

4.4 Java APIs to control Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4.4.1 ThreadGroupController . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4.4.2 AllocationHandler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 OSGi 13

5.1 OSGi concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.2 OSGi architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5.3 OSGi implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5.4 Life Cycle layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5.5 migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

5.5.1 error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

6 High-Level APIs 17

6.1 Bundle resource monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

6.1.1 CPU Budgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

6.1.2 Cost enforcement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

6.1.3 Admission control protocol . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6.1.4 Memory budgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6.2 Extensions to OSGi life cycle managment . . . . . . . . . . . . . . . . . . . . . . . 19

6 November 2014 Version 1.0Confidentiality: EC Distribution

Page iii


7 VM Level APIs 207.1 Dynamic Adding and Removing of CPUs and Frequency Changes . . . . . . . . . . 20

7.2 Freeze and Resume of Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

7.3 Freeze and Resume of whole VM . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7.4 Enhancing freeze and resume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Page iv Version 1.0Confidentiality: EC Distribution

6 November 2014


List of Figures

1 Tasks running on the DreamCloud Framework . . . . . . . . . . . . . . . . . . . . . 2

2 The OSGi framework layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14


Page v


Document Control

Version Status Date0.1 Document outline 1 October 20140.9 Version project partner review 31 October 20141.0 Final review and QA 6 November 2014

Page vi Version 1.0Confidentiality: EC Distribution

6 November 2014


1 Executive Summary

This document constitutes deliverable D4.1 – API Specification of WP4 of the DreamCloud project.This deliverable describes the APIs that can be used to extend the existing functionality of Javaplatform, the underlying OS and the component framework in order to achieve the goals of theproject. The goals include the control over tasks running on the DreamCloud platform and for thegreatest flexibility different levels of control will be provided: from low-level APIs at the OS level toAPIs at the level of OSGi-based component framework up to APIs that control the entire Java VM.


Page 1


2 Background

For the purposes of the implementation DreamCloud tasks are treated as abstract entities scheduledon a networked system.

Within the system software APIs specified in this document, however, we need to decide what basicconcepts are used to implement tasks.

One of the execution environments used in DreamCloud is a Java VM. We must define how to mapDreamCloud tasks onto Java concepts a little weird about mapping tasks to concepts, but ok and todetermine the characteristics of these tasks and how they can be separated from one another. Figure 1provides a high-level overview of a Java VM running several tasks within DreamCloud. These tasksare separated from one another, ensuring that each task exclusively locks the required resources andthose can not be utilized by other tasks.

Java VM (Jamaica)

Task1

DreamCloud Framework

SpecialTask

Task2

RTTask

Task3

Figure 1: Tasks running on the DreamCloud Framework

Java provides an extensive set of features that can serve as containers for tasks, some of these featuresare native to the language or the environment, some are established component frameworks used inthe Java world. Roughly sorted by the level of granularity, the following entities could be used tocontain tasks within DreamCloud:

• Java methods – these are the basic building blocks of Java applications• Java classes or objects – these group several methods and provide a clear interface• java.util.concurrent.RunnableFuture<V> from the Java concurrency APIs pro-

vides a container for an activity that may run in parallel and produce a result value.• java.lang.Thread represents a thread of execution in Java• javax.realtime.RealtimeThread represent threads of execution in realtime Java

([5]) with additional properties such as priorities with realtime semantics.• javax.realtime.AsyncEventHandler represents activities in realtime Java that are

not strictly associated wuth a thread.

Page 2 Version 1.0Confidentiality: EC Distribution

6 November 2014


• Java Xlets [4] provides a framework for components with a simple and clear life cycle modelusing transitions init, start, pause, stop, and destroy.

• JamaicaCAR App is a specific Xlet widely used in app platforms within automotive head units.These components are controlled by a ThreadGroup together with restricted CPU budget,heap memory budget and permission.

• OSGi [8] provides a component model that is applied in a wide variety of applications includinghome automation, IDEs (Eclipse) or the Internet-of-Things.

• java.lang.ThreadGroup provides a mechanism to group several threads• javax.realtime.ProcessingGroup is a realtime Java ([5]) mechanism to control the

scheduling and resource usage of a set of schedulable objects• Java applications could be treated as tasks within DreamCloud.

After careful analysis of requirements (see next section 3), it was decided not to rely on one singlemechanism for DreamCloud, but to provide APIs at different levels for tasks of different granularities.An important role is allocated to OSGi bundles that provide a mature component model DreamCloudcan use as a basis (see section 5). However this model has to be extended and new mechanisms needto be provided for the migration of components (section 6).

An aspect that is orthogonal to the component model is the extent to which the DreamCloud schedul-ing and task migration may remain transparent to the developer of the components. Different levels ofsupport (support for what?) are discussed in section 7 that describes the migration support providedby the Java VM.


Page 3


3 Requirements

In this section, we review the requirements defined in deliverable D1.2 – Dynamic Resource Allo-cation Requirements [14] and map those that affect the definitions of APIs in the system softwareinfrastructure to the specific sections of this document.

• Req 3.3.1 Objectives of dynamic resource management should be configurableThis requirement has no direct effect on the system software APIs.Specific objectives, however, will indirectly result in requirements on the system software in-frastructure. A good candidate is energy efficiency, that requires APIs for dynamically addingand removing CPUs detailed in section 7.1. Meeting realtime deadlines requires APIs forcost enforcement (sections 4.4.1, 6.1.2), admission control (section 6.1.3), and guaranteeingmemory budgets require APIs to enforces tasks to stay within their budgets (sections 4.4.2and 6.1.4).

• Req 3.3.2 Specified hard real-time constraints shall not be violatedThis requirement prescribes for the system software to provide realtime guaranteesThis does not affect any new APIs defined in this document, but it requires the underlyingJava implementation and operating system to provide the realtime guarantees as detailed insection 4.1.

• Req 3.3.3 Dynamic resource allocation shall be used to provide different levels of perfor-mance guaranteesThis requirement demands the availablity of APIs for management of system level resourcesThe system infrastructure must provide APIs for the resources managed by the Java VM andthe OS, in particular CPUs (section 4.1 and 7.1) and memory (sections 4.4.2 and 6.1.4).

• Req 3.3.4 The average latency of jobs shall be minimisedThis requirement translates into performance requirements for the system infrastructure.In particular, it would not be sufficient to execute Java tasks by a Java bytecode interpretersince this would probably result in unacceptably poor runtime performance compared to non-DreamCloud. As a result, the system software will require mechanisms such as compilers toenhance the performance (section 4.3).

• Req 3.3.5 The total energy dissipation of jobs shall be minimisedThis requirement requests the APIs for controlling resources that cause energy dissipation suchas CPU and memory usage.This is related to the resource allocation requirement 3.3.3 since controlling the usage of CPUs(section 4.1 and 7.1) and memory (sections 4.4.2 and 6.1.4) has a direct effect on the energyconsumption. In particular, disabling unused CPUs (section 7.1) or reducing the clock fre-quency (section 4.2) gives some control over the energy dissipation.

• Req 3.3.6 Communication overhead parameters shall be predictableThis requirement calls for general realtime guarantees in the system software infrastructureFrom the Java VM and OS perspective, this requirement is covered by requirement 3.3.2, real-time guarantees (section 4.1).

• Req 3.3.7 Dynamic resource allocation overhead shall be predictable and boundedThis requirement affects all APIs defined within the system infrastructure.The time required for all APIs provided by the system software infrastructure and used for dy-namic resource allocation, must provide predictable and bounded overhead. (not clear what


6 November 2014


"required" refers to, please clarify) This overhead should be very small, in the order of µ-seconds, for monitoring and controlling memory and CPU usage (sections 4.1 and 6.1.4), inthe order of milli-seconds for enabling or disabling CPUs or changing their frequencies (sec-tions 4.2 and 7.1), and in the order of 10-100s of milli-seconds for the migration of larger tasksbetween computational nodes (section 7.2).

• Req 3.3.8 The dynamic resource allocation mechanisms shall cope with dynamic work-loadThis requirement applies to the resource allocation mechanism, not to the system softwareinfrastructure APIs.The APIs defined in this deliverable are not affected by this.

• Req 3.3.9 The dynamic resource allocation mechanisms shall not limit hardware scalingThis requirement applies to the resource allocation mechanism, not to the system softwareinfrastructure APIs.The APIs defined in this deliverable are not affected by this.

• Req 3.3.10 The dynamic resource allocation mechanisms shall cope with limited informa-tion about the state of the overall systemThis requirement applies to the resource allocation mechanism, not to the system softwareinfrastructure APIs.The APIs defined in this deliverable are not affected by this.

• Req 3.3.11 The dynamic resource allocation mechanisms shall respect mapping con-straints that restrict the allowed computational unitThis requirement applies to the resource allocation mechanism, not to the system softwareinfrastructure APIs.The APIs defined in this deliverable are not affected by this.

• Req 3.3.12 The dynamic resource allocation mechanisms shall consider cost, runtime andpower efficiency for different type of resources available to a multi-typed jobThis requirement applies to the resource allocation mechanism, not to the system softwareinfrastructure APIs.The system software infrastructure, however, can help to increase the number of resources thatcan be used to perform a given job: The use of different Java virtual machines on different HWmay allow the same job to run on very different systems using, e.g., ARM and x86/64 CPUs oreven accelerators such as FPGAs (section 4.1.1).

Table 1 gives a compact overview of the requirements that are addressed by the APIs defined in thesections of this document.


Page 5


Requirement Section(s)Req 3.3.1 Objectives of dynamic resource managementshould be configurable

7.1 4.4.1 6.1.2)6.1.3 4.4.2 6.1.4

Req 3.3.2 Specified hard real-time constraints shall not beviolated

4.1

Req 3.3.3 Dynamic resource allocation shall be used to pro-vide different levels of performance guarantees

4.1 7.1 4.4.2 6.1.4

Req 3.3.4 The average latency of jobs shall be minimised 4.3Req 3.3.5 The total energy dissipation of jobs shall be min-imised

4.1 7.1 4.4.2 6.1.47.1 4.2

Req 3.3.6 Communication overhead parameters shall bepredictable

4.1

Req 3.3.7 Dynamic resource allocation overhead shall bepredictable and bounded

4.1 6.1.4 4.2 7.17.2

Req 3.3.8 The dynamic resource allocation mechanismsshall cope with dynamic workload

–

Req 3.3.9 The dynamic resource allocation mechanismsshall not limit hardware scaling

–

Req 3.3.10 The dynamic resource allocation mechanismsshall cope with limited information about the state of theoverall system

–

Req 3.3.11 The dynamic resource allocation mechanismsshall respect mapping constraints that restrict the allowedcomputational unit

–

Req 3.3.12 The dynamic resource allocation mechanismsshall consider cost, runtime and power efficiency for differ-ent type of resources available to a multi-typed job

4.1.1

Table 1: Requirements addressed by System Software APIs in this document.


6 November 2014


4 Low-Level Resource Control

This section defines a set of low-level APIs for resource control that can be used by higher levelframeworks or applications. Particular focus is given to the OSGi framework that will be describedin following sections 5 and 6.

This section lists Java VM requirements necessary to provide realtime features and high performancethat can be fulfilled by the existing realtime Java implementation provided by Jamaica, as well asextensions that allow additional control of Threads, CPU usage and memory allocation.

4.1 Realtime Java and OS

4.1.1 Base Java environment

A fundamental prerequisite for the system software infrastructure is to provide realtime guaran-tees. For the Java implementation, this means ability to support the Realtime Specification for Java(RTSJ) as specified in the JSR 282 [5] and implement concurrent and parallel realtime garbage col-lection [15]. Realtime APIs also include fine-grained control over scheduling decisions such as CPUaffinities provided by the RTSJ through class javax.realtime.Affinity and APIs related tothis class.

Usage of a Java VM makes it possible to run the same job on different hardware, like ARM or x86/64CPUs with different numbers of cores. One might even go further and use the Java VM on top ofhardware accelerators, for example FPGA-based ones, as demonstrated by the JUNIPER projects [9].

4.1.2 Base OS environment

At the OS level, support for realtime priorities is provided, for instance via the POSIX schedul-ing policy SCHED_FIFO [13]. Control over CPU affinities can be obtained via the Linux-specificsched_setaffinity system call.

4.2 Controlling CPU frequency

The energy dissipation of today’s hardware depends mostly on the frequencies of the underlyingCPUs. In order to match the requirement Req 3.3.5 The total energy dissipation of jobs shall beminimised from D1.2 ([14]), we might need certain means to control these frequencies. This istypically done at the OS level. On Linux CPU frequency can be controlled for each individual CPUusing the sysfs mechanism [10].

Although Linux command cpufreq-set [6] provides a more convenient way to control the frequency,it essentially performs direct writes to the same sysfs cpufreq interface. (here I’m a little lost: is thereany conclusion from the whole subsection?)


Page 7


4.3 Performance Optimization and Compilation

The use of a Java VM within DreamCloud brings up the question on how the performance require-ments (Req 3.3.4 The average latency of jobs shall be minimised from D1.2 cited1.2) can be achieved.The Java VM specification [11] defines a virtual machine that can be implemented either as a simpleinterpreter, or the one that uses different compilation techniques for better performance. The compi-lation techniques usually rely on either just-in-time compilation, as provided by OpenJDK [7], or onstatic ahead-of-time compilation as provided by JamaicaVM [1].

In the context of DreamCloud, a simple interpreter implementation provides the fastest and the mostflexible results, in particular considering the dynamic migration features described in section 7. Thejust-in-time compilation approach results in very unpredictable timing and can therefore not be ap-plied within DreamCloud. The ahead-of-time compilation, however, provides high performance andpredictable timing, but is harder to apply to dynamic systems such as OSGi. It also typically doesnot allow migration between different hardware architectures.

Within DreamCloud, Jamaica ahead-of-time compiler will therefore be extended to allow the com-pilation of single bundles. A new tool –the bundler– will provide the ability to equip a component(OSGi bundle, xlet, etc.) provided as a Java jar file with pre-compiled machine code for one orseveral target platforms.

4.4 Java APIs to control Threads

Two classes are to be added to provide additional control over the resource usage by threadsor groups of threads that belong to one component (OSGi bundle, Xlet, etc.). A lim-ited control is already provided via standard Java class java.lang.ThreadGroup [2]and by RTSJ [5, 3] classes javax.realtime.ProcessingGroupParameters andjavax.realtime.MemoryParameters.

4.4.1 ThreadGroupController

Class ThreadGroupController permits low-level control over the CPU usage and the numberof threads running within one ThreadGroup. An instance of ThreadGroupController shouldbe made accessable from an instance of ThreadGroup via a call to the static function get. Thisapproach enables low-level control over the threads in that group.

/ * ** T h i s c l a s s p e r m i t s e x e c u t i o n t i m e c o n t r o l over t h e members o f a* ThreadGroup .* /

p u b l i c c l a s s T h r e a d G r o u p C o n t r o l l e r{

/ * ** C l a s s t h a t p e r m i t s t o d e f i n e a wrapper method around a t h r e a d ’ s* run method .* /


6 November 2014


p u b l i c s t a t i c a b s t r a c t c l a s s RunWrapper{

/ * ** wrapper method t h a t has t o c a l l r . run ( ) .** @param r t h e r u n n a b l e .* /

p u b l i c a b s t r a c t vo id c a l l R u n ( Runnable r ) ;

}

/ *−−−−−−−−−−−−−−−−−−−−−−−−−−−−− methods −−−−−−−−−−−−−−−−−−−−−−−−−−−−−* /

/ * ** g e t o b t a i n s t h e T h r e a d G r o u p C o n t r o l l e r f o r a g i v e n ThreadGroup .* I f no c o n t r o l l e r was i n s t a l l e d y e t , t h i s method w i l l c r e a t e one .** @param g t h e ThreadGroup** @return t h e c o n t r o l l e r , n e v e r n u l l .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and a* c a l l t o i t s <code >checkAcces s </ code > method w i t h t h e c o n t r o l l e d* <code >ThreadGroup </ code > as i t s argument r e s u l t s i n t h r o w i n g a* <code >S e c u r i t y E x c e p t i o n </ code >.* /

p u b l i c s t a t i c T h r e a d G r o u p C o n t r o l l e r g e t ( ThreadGroup g ) ;

/ * ** I n s t a l l a wrapper t o be c a l l e d around t h e run ( ) method f o r a l l* newly s t a r t e d t h r e a d s i n t h i s t h r e a d group .** @param r t h e run wrapper , must n o t be n u l l** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and a* c a l l t o i t s <code >checkAcces s </ code > method w i t h t h e c o n t r o l l e d* <code >ThreadGroup </ code > as i t s argument r e s u l t s i n t h r o w i n g a* <code >S e c u r i t y E x c e p t i o n </ code >.* /

p u b l i c vo id se tRunWrapper ( RunWrapper r ) ;

/ * ** s e t M a x A c t i v e C o u n t s e t s t h e maximum number o f a c t i v e t h r e a d s* a l l o w e d f o r t h i s ThreadGroup .** I f t h e number o f a c t i v e t h r e a d s i s >= max , no new t h r e a d s w i l l be* p e r m i t t e d f o r t h i s ThreadGroup . Threads c r e a t e d i n t h e p a s t are* u n a f f e c t e d i f max i s s e t lower l a t e r .


Page 9


** @param max t h e maximum number o f a c t i v e t h r e a d s .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and a* c a l l t o i t s <code >checkAcces s </ code > method w i t h t h e c o n t r o l l e d* <code >ThreadGroup </ code > as i t s argument r e s u l t s i n t h r o w i n g a* <code >S e c u r i t y E x c e p t i o n </ code >.* /

p u b l i c vo id se tMaxAct iveCoun t ( i n t max ) ;

/ * ** g e t C y c l e s o b t a i n s t h e number o f CPU c y c l e s t h a t a l l t h r e a d s i n* t h i s group have run so f a r .** @return t h e number o f c y c l e s .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and a* c a l l t o i t s <code >checkAcces s </ code > method w i t h t h e c o n t r o l l e d* <code >ThreadGroup </ code > as i t s argument r e s u l t s i n t h r o w i n g a* <code >S e c u r i t y E x c e p t i o n </ code >.* /

p u b l i c long g e t C y c l e s ( ) ;

/ * ** s e t M a x P r i o r i t y s e t s t h e maximum p r i o r i t y o f t h i s t h r e a d group and* o f a l l i t s c h i l d r e n . <p>** U n l i k e ThreadGroup . s e t M a x P r i o r i t y , t h i s f u n c t i o n a l s o r e d u c e s t h e* p r i o r i t y o f any t h r e a d s i n t h i s group t h a t may a l r e a d y have a* h i g h e r p r i o r i t y . However , i t does n o t change t h e p r i o r i t y o f* r u n n i n g t h r e a d s o f any c h i l d r e n o f t h i s t h r e a d group .** @param p r i t h e new maximum p r i o r i t y . A p r i o r i t y v a l u e o f "0" can* be used t o f o r c e a l l t h r e a d s t o Java p r i o r i t y "1" w i t h Jamaica ’ s* S c h e d u l e r . m i c r o A d j u s t P r i o r i t y ( ) o f "−1" such t h a t t h e s e t h r e a d s* w i l l n e v e r run i f t h e r e i s a h i g h e r p r i o r i t y th read , even i n t h e* non− s t r i c t p r i o r i t y range .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and a* c a l l t o i t s <code >checkAcces s </ code > method w i t h t h e c o n t r o l l e d* <code >ThreadGroup </ code > as i t s argument r e s u l t s i n t h r o w i n g a* <code >S e c u r i t y E x c e p t i o n </ code >.* /

p u b l i c vo id s e t M a x P r i o r i t y ( i n t p r i ) throws S e c u r i t y E x c e p t i o n ;

/ * ** For a l l t h r e a d s i n t h i s t h r e a d group t h a t are b l o c k e d i n an I /O* f u n c t i o n , t r y t o i n t e r r u p t t h i s I /O f u n c t i o n . T h i s o n l y works* f o r I /O o p e r a t i o n s t h a t are i n t e r r u p t i b l e . C u r r e n t l y ,* i n t e r r u p t i b l e I /O o p e r a t i o n s i n c l u d e


6 November 2014


** S e r v e r t S o c k e t . a c c e p t ( )* F i l e I n p u t S t r e a m . read ( )* F i l e O u t p u t S t r e a d m . w r i t e ( )** I n t e r r u p t i n g I /O may c l o s e t h e u n d e r l y i n g f i l e or s o c k e t and* r e p l a c e i t by a d i f f e r e n t one . E . g . , i f a t h r e a d b l o c k e d r e a d i n g* from Sys tem . i n or w r i t i n g t o Sys tem . o u t w i l l be i n t e r r u p t e d ,* Sys tem . i n / o u t w i l l a f t e r w a r d s be u n a v a i l a b l e f o r any o t h e r* t h r e a d .** Threads o f any c h i l d r e n o f t h i s group w i l l n o t be a f f e c t e d .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and a* c a l l t o i t s <code >checkAcces s </ code > method w i t h t h e c o n t r o l l e d* <code >ThreadGroup </ code > as i t s argument r e s u l t s i n t h r o w i n g a* <code >S e c u r i t y E x c e p t i o n </ code >.* /

p u b l i c vo id i n t e r r u p t B l o c k i n g I O ( ) ;

}

4.4.2 AllocationHandler

Class AllocationHandler permits the installation of a handler called on every allocation per-formed by a given thread. This handler can independently decide whether to disallow allocation orto trace it, e.g., via instances of PhantomReference, etc./ * *

* A l l o c a t i o n H a n d l e r p r o v i d e s a means t o a t t a c h a h a n d l e r t o a t h r e a d* t h a t w i l l be i n v o k e d e v e r y t i m e t h e t h r e a d p e r f o r m s an a l l o c a t i o n .* /

p u b l i c a b s t r a c t c l a s s A l l o c a t i o n H a n d l e r{

/ *−−−−−−−−−−−−−−−−−−−−−−−−−−−− methods −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−* /

/ * ** s e t t h e a l l o c a t i o n h a n d l e r f o r a g i v e n t h r e a d** @param t t h e t h r e a d** @param h t h e a l l o c a t i o n hand le r , must r e d e f i n e method ha nd l e ( O b j e c t )** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" A l l o c a t i o n H a n d l e r . s e t " ) </ t t >* i n s t a l l a t i o n o f an a l l o c a t i o n h a n d l e r .* /

p u b l i c s t a t i c synchronized void s e t A l l o c a t i o n H a n d l e r ( Thread t ,A l l o c a t i o n H a n d l e r h ) ;


Page 11


/ * ** Remove t h e g i v e n memory h a n d l e r .** @param t t h e t h r e a d t h e h a n d l e r was a s s i g n e d t o** @param h t h e a l l o c a t i o n h a n d l e r** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" A l l o c a t i o n H a n d l e r . remove ") </ t t >* removal o f an a l l o c a t i o n h a n d l e r .* /

p u b l i c s t a t i c synchronized void r e m o v e A l l o c a t i o n H a n d l e r ( Thread t ,A l l o c a t i o n H a n d l e r h ) ;

/ * ** R o u t i n e t h a t w i l l be c a l l e d when a t h r e a d a l l o c a t e d an o b j e c t .** @param o t h e a l l o c a t e d o b j e c t** @return t h e a l l o c a t e d o b j e c t o or n u l l i f t h e a l l o c a t i o n s h o u l d* f a i l .* /

p u b l i c a b s t r a c t O b j e c t h a n d l e ( O b j e c t o ) ;

/ * ** R o u t i n e t h a t w i l l be c a l l e d b e f o r e a t h r e a d a t t e m p t s t o a l l o c a t e an o b j e c t .* T h i s r o u t i n e may d i s a l l o w an a l l o c a t i o n by r e t u r n i n g f a l s e .** On t h e a l l o c a t i o n o f a m u l t i−d i m e n s i o n a l array , e . g . , new i n t [ 3 ] [ 4 ] , t h i s* method w i l l be c a l l e d on t h e a l l o c a t i o n o f e v e r y s i n g l e −d i m e n s i o n a l a r r a y* s e p a r a t e l y , e . g . once w i t h < t t >a l l o w ( i n t [ ] [ ] . c l a s s , 3 ) < / t t > and t h r e e t i m e s* < t t >a l l o w ( i n t [ ] . c l a s s , 4 ) < / t t >.** The d e f a u l t i m p l e m e n t a t i o n a lways r e t u r n s t r u e .** @param c l a s s t h e c l a s s o f t h e o b j e c t t o be a l l o c a t e d .** @param l e n g t h t h e l e n g t h o f t h e a r r a y t h a t i s t o be a l l o c a t e d i n case c l a s s* i s an array , −1 o t h e r w i s e .** @return t r u e i f f a l l o c a t i o n s h o u l d be a l lowed , f a l s e i f i t s h o u l d be* d e c l i n e d* /

p u b l i c boolean a l l o w ( C l a s s c l a z z ,i n t l e n g t h ) ;

}


6 November 2014


5 OSGi

In this section the OSGi framework is introduced as the system of executable entities to build upon.The desired real-time and resource monitoring features are implemented as extensions to the basicOSGi framework. The basic terminology and concepts such as bundes, the bundle life cycle andresource management are explained and then extended. Furthermore, an API that enables migrationof bundles is proposed.

5.1 OSGi concepts

The OSGi framework [8] gives the proposed answer to the question of what the Java representation ofan executable entity ("runnable") looks like: An executable entity is represented by an OSGi module,a so called bundle. OSGi is a specification for a dynamic module system for Java.

The key features of an OSGi implementation are:

Modularity OSGi embodies a component-based model where modules are dynamically installed andloaded. A bundle hides its internals from other bundles and communicates only via well-defined services.

Dependency management Each module clearly specifies its dependencies in its manifest.mf file.Dependency management of bundles, including semantic versioning, is provided.

Distribution A bundle explicitly registers the services that it provides or requires in order to run.Robustness Each bundle may be stopped or uninstalled from the framework at any moment. Since

each bundle may provide a service, bundles may cooperate with each other. If a bundle pro-viding a service is stopped, uninstalled or fails, the OSGi framework provides a mechanismto safely handle the bundle removal by deactivating the bundle listeners. Once this action isperformed, all the bundles that were using the services provided by the removed bundle areautomatically notified by the framework.

Other notable features are:

Security The OSGi security layer is built on top of the Java security layer and extends it by providinga fine-grained model which can be executed at all OSGi layers. The bundle developer mayspecify the requested security details for one or more layers and implement a security policyfor those. For example, it is possible to apply a security policy to the bundles by applying aJAR signature and thus limiting the number of authorized users.

Evolution As mentioned previously, a bundle can make its dependencies available through the di-rectives Import-Package and Export-Package. OSGi also supports versioning as part of de-pendency management. OSGi allows for multiple versions of a Java package to be present atruntime simultaneously.

5.2 OSGi architecture

In order to meet the goals of the DreamCloud project, a real-time version of the OSGi framework isproposed.In order to introduce the required extensions that add real-time capabilities to the frame-work, it is necessary to describe the underlying concepts of OSGi and real-time systems in general.


Page 13


The functionality of the OSGi framework is divided in six layers, as shown in Figure 2.

Figure 2: The OSGi framework layers

A brief introduction to the OSGi layers as follows:

Bundles: the OSGi components provided by the developers.Services: connect bundles in a dynamic way by offering a publish-find-bind model.Life Cycle: provides the API to install, start, stop, update and uninstall bundles.Modules: defines how a bundle can import and export other bundles’ packages.Security: handles the security aspects, it can be applied to all OSGi layers.Execution Environment: defines what methods and classes are available in a specific platform.

.

5.3 OSGi implementations

It is not possible to extend the OSGi architecture with the proposed APIs without implementing thesefor a particular OSGi implementation. Hence a choice needs to be made.

There are lots of different OSGi implementations available:

• Apache Felix• Concierge• Knopflerfish• Equinox (Eclipse).

Out of these Concierge was selected, as it has the lowest footprint and can easily be extended withthe required features such as bundle migration or enforcement of real-time requirements.


6 November 2014


5.4 Life Cycle layer

The Life Cycle layer adds bundles that can be dynamically installed, started, stopped, updated anduninstalled. Bundles rely on the OSGi implementation for class loading but need to implement theBundleActivator interface so that they can be managed at run time.

Bundle State DescriptionINSTALLED The bundle has been successfully installed.RESOLVED All Java classes that the bundle needs are available. This

state indicates that the bundle is either ready to be started orhas stopped.

STARTING The bundle is being started, the BundleActivator.startmethod will be called, and this method has not yet returned.When the bundle has an activation policy, the bundle willremain in the STARTING state until the bundle is activatedaccording to its activation policy.

ACTIVE The bundle has been successfully activated and is running;its Bundle Activator start method has been called and re-turned.

STOPPING The bundle is being stopped. The BundleActivator.stopmethod has been called but the stop method has not yet re-turned.

UNINSTALLED The bundle has been uninstalled. It cannot move into an-other state.

5.5 migration

It is natural to extend the life cycle layer with two states PAUSED and RESUMED so that bundlemigration is supported.

On the API side however, there are lots of possibilities to implement the bundle migration. Wepropose an extension to the BundleActivator API:

/ * E x i s t i n g OSGi i n t e r f a c e : * /i n t e r f a c e B u n d l e A c t i v a t o r{

void s t a r t ( Bund l eCon tex t c o n t e x t ) throws E x c e p t i o n ;void s t o p ( Bund leCon tex t c o n t e x t ) throws E x c e p t i o n ;

}

/ * ** Proposed e x t e n s i o n* /

i n t e r f a c e M i g r a t a b l e B u n d l e A c t i v a t o r ex tends B u n d l e A c t i v a t o r{

S e r i a l i z a b l e pause ( Bund leCon tex t c o n t e x t ) throws E x c e p t i o n ;void resume ( Bund leCon tex t c o n t e x t , S e r i a l i z a b l e s t a t e ) throws E x c e p t i o n ;

}


Page 15


It is important to note that this proposal is completely independent of the VM extensions discussedin section 7.2.

The pausemethod is used to serialize any state required by the bundle into an object. This serializedobject then is transferred to the other node that will continue to run the bundle. The bundle then willbe started with the help of the resume method that receives the serialized object.

If a bundle migration occurs, the methods start and stop are not invoked, so the chain is start-> pause -> resume -> stop where start and pause happen on the node that is mi-grated from and resume and stop occurs on the node that the bundle is migrated to.

5.5.1 error handling

If pause fails by throwing some exception, the bundle shall not be paused or stopped in any way.The idea is that a criticial service is currently running and cannot be paused. Instead the exceptioncan be logged or a message can be shown to the user that the request to pause the bundle failed.

The user may then chose whether to pause the bundle later or stop the bundle instead.

If resume fails by throwing some exception, it indicates a fatal error and the bundle state cannot beresumed. However, since the state has been serialized, it is safe to retry the resume operation onthe same or on a different node.


6 November 2014


6 High-Level APIs

This section describes how the OSGi architecture is extended with the required real-time and moni-toring features.

6.1 Bundle resource monitoring

The Bundle Resource Monitoring mechanism ensures that resources of each bundle can be monitoredseparately. In order to implement this feature, a thread group is allocated to each bundle and all thethreads spawned in that bundle must belong to that bundle thread group. Typical OSGi frameworkimplementations, such as Concierge, do not implement this feature since it is not part of the OSGispecification. If this mechanism is not provided by the OSGi Framework implementation then upona start of a bundle containing threads and/or thread groups, all the nested threads and thread groupswill be spawned as childr processes of "main" Java ThreadGroup. This thread group is the defaultone created by the Java runtime system upon a start up of the application.

Such behavior is problematic since there is a need to distinguish between the threads spawned indifferent bundles. The work of Miettinen et al. [12]) on OSGi resources monitoring proposed anapproach to deal with the issue. The author states that in order to achieve a bundle-level separation,it is necessary is to identify threads and thread groups that are associated with each bundle. Inthis way, the OSGi application execution is separated in different bundle-associated threads. Oncethis separation is achieved, it becomes easy to monitor all the threads associated with a bundle.This mechanism is necessary to implement many features implemented in this contribution such asMemory Limits Enforcement and Forcefully Thread Termination.

6.1.1 CPU Budgets

CPU budgets are per bundle / thread group. A CPU budget is defined by the following attributes:

Period The fixed phases in which the running time is subdivided.

Cost The CPU budget of the group, namely its processing time avaliable per period.

Start Time at which the first period begins.

Deadline The latest permissible completion time measured from the start of the current period.

All these attributes are measured in miliseconds.

6.1.2 Cost enforcement

The cost enforcement is a mechanism that enables the application to recover from an overrun when-ever cost monitoring fires an event,. There are several approaches to ensure that the applicationrecovers from an overrun, but they may require the application to be designed to be interruptable orto poll for an interruption. For the purposes of this project, the recovery function is designed in such


Page 17


a way that it does not require any interference on the user’s side. The logic of the recovery functionis to reduce the priorities of the overrunning threads to a lower value, so that other threads can makeprogress. Therefore, the cost enforcement implementation relies on the thread priority level: it doesnot stop threads, but simply lowers or raises their priorities.

On a cost overrun event, the cost overrun handler is executed lowering the priorities of every threadbelonging to the thread group of the bundle.

A periodic timer fires cost replenishment events. These are handled by the replenishment handlerthat then raises the priorities of these threads again.

The entire process that describes the cost monitoring and enforcement functionality is called budgetreplenishment mechanism.

6.1.3 Admission control protocol

Admission control protocol (ACP) acts as a filter; whenever a start of a bundle is requested, anacceptance test is performed in order to establish if the activation of the bundle is feasible or not.

The main purpose of the protocol is to control the load of the CPU in terms of the number of bundlesthat are simultaneously started in the OSGi framework. It may be unclear why providing an admissioncontrol protocol for the OSGi Framework since the RTSJ already provides admission control forSchedulable objects, namely RealtimeThreads and AsyncEventHandlers.

It is technically possible to assign certain parameters to Schedulable objects in order to specify theirtemporal requirements in terms of costs, start times, periods and deadlines. However, the RTSJspecification only states that this information can be used to prevent the admission of a schedulableobject if the resulting system would not be feasible from a scheduling perspective.However thisapproach does not provide any detail about how to implement it. Since the OSGi Framework withreal-time enhancements runs on JamaicaVM and the default feasibility analysis done by this real-time virtual machine cannot be applied to OSGi bundles, we do not to rely on the VM schedulingfeasibility analysis. For this reason, a separate admission control protocol, designed specifically forOSGI bundles, needs to be implemented.

Since only the starting and updating phases of the life cycle can increase the load of a system, theACP needs to be implemented just for these parameters:

• Starting: The start of a bundle that fails the acceptance test of ACP is denied. The acceptancetest makes use of the values cost (the computational time) and the replenishment period thatare provided with the bundle. If its activation is denied, the bundle has to wait until some otherbundle in the start state is stopped and its resources are freed.

• Updating: Since updates are modelled as start+stop the action taken for "starting" suffices.

6.1.4 Memory budgets

There are two attributes that can be used to configure the enforcement of memory limits:

MIN_FREE_ALARM minimum free available memory before raising an alarm to the user. If thismemory value is reached, the platform generates a warning to the user.


6 November 2014


MIN_FREE_OUT_OF_MEMORY represents the minimum free available memory before an action istaken. The bundle is stopped to avoid an OutOfMemoryError. If this value is set to 0 then noaction is taken.

The RTSJ functionality called Asynchronous Transfer of Control is used to stop every thread in thebundle as soon as MIN_FREE_OUT_OF_MEMORY is reached.

6.2 Extensions to OSGi life cycle managment

To implement the mentioned features, the standard OSGi life cycle needs to be extended. In particu-lar, the following list describes the actions that are taken for each bundle’s life cycle state.

Starting • First the bundle needs to be assigned a budget: The budget configuration file is parsed,assigns its parameters to the bundle and starts the replenishment periodic timer.

• Then the admission control needs to check the feasibility of the bundle.• For starting the bundle the resource monitoring assigns a ThreadGroup to the bundle.

Stopping • The forceful thread termination calls the fire() method to notify threads and real-time threads to be safely interrupted.

• The replenishment periodic timer for the bundle is disabled.• The cost overrun handler for the bundle is disabled.

Active • During the runtime of a bundle cost monitoring and enforcement need to run continu-ously.

• Likewise every single allocation could mean that the bundle uses more memory thanallowed by its limit and so needs to be tracked by the memory limits enforcement mech-anism.

Updating • The admission control needs to check the feasibility of the bundle.


Page 19


7 VM Level APIs

Unlike the low-level APIs described in section 4, the APIs specified at the VM level are meant toprovide functions of external control for the VM. Nevertheless, these APIs are provided as Java APIs.These APIs are to be used by specific frameworks that communicate with the external scheduler inorder to perform the required actions on the VM.

These actions in particular are: disabling and enabling of CPUs as well as (section 7.1) freezing andresuming of tasks (section 7.2.

7.1 Dynamic Adding and Removing of CPUs and Frequency Changes

The dynamic addition and removal of CPUs poses a significant challenge for the realtime schedulerof the Java virtual machine. It has to be ensured that communication between threads running ondifferent CPUs would not suffer from CPUs being disabled. That means a thread that performsan flip operation to start a new garbage collection cycle [15] performs a busy loop to wait for allother CPUs to finish all heap modifications such as write-barriers executed on reference assignments.Disabling of a CPU must result in any thread being blocked until this CPU is re-enabled.

Therefore, the APIs require notification before hardware CPUs are to be stopped and, vice versa,hardware CPUs must be restarted before the Java VM can be allowed to use these CPUs again.

Low-level APIs from the underlying operating system enable dynamic modification of the CPU fre-quencies (see section 4.2). To abstract from the underlying OS, it will help to have correspondingVM level APIs in Java. Therefore, the VM level APIs should include APIs to modify the CPU fre-quencies.

The following listing provides the API specification and JavaDoc documentation of the CPU controlAPIs proposed by DreamCloud.

import j a v a . u t i l . B i t S e t ;import j a v a . u t i l . S e t ;

/ * ** CpuContro l p r o v i d e s means t o e n a b l e or d i s a b l e CPUs f o r t h e use by t h e Java* a p p l i c a t i o n .* /

p u b l i c c l a s s CpuCont ro l{

/ *−−−−−−−−−−−−−−−−−−−−−−−−−−−− methods −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−* /

/ * ** o b t a i n t h e s e t o f p r o c e s s o r s t h a t are a v a i l a b l e f o r t h e Java VM t o e x e c u t e* Java t h r e a d s , i n d e p e n d e n t o f whe ther t h e y are e n a b l e d or d i s a b l e d .** @return t h e s e t o f a v a i l a b l e CPUs s t o r e d i n a newly a l l o c a t e d i n s t a n c e o f* B i t S e t .* /


6 November 2014


p u b l i c s t a t i c B i t S e t a l l C p u s ( ) ;

/ * ** o b t a i n t h e s e t o f p r o c e s s o r s t h a t are a v a i l a b l e f o r t h e Java VM t o e x e c u t e* Java t h r e a d s , i n d e p e n d e n t o f whe ther t h e y are e n a b l e d or d i s a b l e d .** @param d e s t a b i t s e t t o r e c e i v e t h e s e t o f a v a i l a b l e CPUs . May* be n u l l t o ask f o r a l l o c a t i o n o f a new b i t s e t .** @return t h e s e t o f a v a i l a b l e CPUs . In case d e s t != n u l l , r e s u l t w i l l be e q u a l* t o d e s t , o t h e r w i s e i t w i l l be a newly a l l o c a t e d i n s t a n c e o f B i t S e t .* /

p u b l i c s t a t i c B i t S e t a l l C p u s ( B i t S e t d e s t ) ;

/ * ** o b t a i n t h e s e t o f p r o c e s s o r s t h a t are c u r r e n t l y e n a b l e d f o r t h e Java VM t o* e x e c u t e Java t h r e a d s .** @return t h e s e t o f e n a b l e d CPUs s t o r e d i n a newly a l l o c a t e d i n s t a n c e o f* B i t S e t .* /

p u b l i c s t a t i c B i t S e t enab ledCpus ( ) ;

/ * ** o b t a i n t h e s e t o f p r o c e s s o r s t h a t are c u r r e n t l y e n a b l e d f o r t h e Java VM t o* e x e c u t e Java t h r e a d s .** @param d e s t a b i t s e t t o r e c e i v e t h e s e t o f e n a b l e d CPUs . May* be n u l l t o ask f o r a l l o c a t i o n o f a new b i t s e t .** @return t h e s e t o f e n a b l e d CPUs . In case d e s t != n u l l , r e s u l t w i l l be e q u a l* t o d e s t , o t h e r w i s e i t w i l l be a newly a l l o c a t e d i n s t a n c e o f B i t S e t .* /

p u b l i c s t a t i c B i t S e t enab ledCpus ( B i t S e t d e s t ) ;

/ * ** o b t a i n t h e s e t o f p r o c e s s o r s t h a t are c u r r e n t l y d i s a b l e d f o r t h e Java VM t o* e x e c u t e Java t h r e a d s .** @return t h e s e t o f d i s a b l e d CPUs s t o r e d i n a newly a l l o c a t e d i n s t a n c e o f* B i t S e t .* /

p u b l i c s t a t i c B i t S e t d i s a b l e d C p u s ( ) ;

/ * ** o b t a i n t h e s e t o f p r o c e s s o r s t h a t are c u r r e n t l y d i s a b l e d f o r t h e Java VM t o* e x e c u t e Java t h r e a d s .*


Page 21


* @param d e s t a b i t s e t t o r e c e i v e t h e s e t o f d i s a b l e d CPUs . May* be n u l l t o ask f o r a l l o c a t i o n o f a new b i t s e t .** @return t h e s e t o f d i s a b l e d CPUs . In case d e s t != n u l l , r e s u l t w i l l be e q u a l* t o d e s t , o t h e r w i s e i t w i l l be a newly a l l o c a t e d i n s t a n c e o f B i t S e t .* /

p u b l i c s t a t i c B i t S e t d i s a b l e d C p u s ( B i t S e t d e s t ) ;

/ * ** d i s a b l e t h e CPUs t h a t are s p e c i f i e d by t h e g i v e n B i t S e t . Any Java t h r e a d s* c u r r e n t l y r u n n i n g on any o f t h e s e CPUs w i l l be preempted and p u t i n t o READY* s t a t e or moved t o a n o t h e r a v a i l a b l e CPU i f t h e i r a f f i n i t y s e t p e r m i t s and* o t h e r CPUs are a v a i l a b l e . The Jamaica s c h e d u l e r w i l l u n s c h e d u l e a l l t a s k s* on any o f t h e s e CPUs b e f o r e t h i s c a l l r e t u r n s .** A f t e r t h i s c a l l , < t t >enabledCpus ( ) < / t t > w i l l e q u a l o l d* < t t >enabledCpus ( ) . andNot ( cpus ) </ t t >.** Any t h r e a d s w i t h an { @link j a v a x . r e a l t i m e . A f f i n i t y } t h a t i s d i s j o i n t w i t h* t h e new < t t >enabledCpus ( ) < / t t > w i l l no l o n g e r be a b l e t o run .** A f t e r t h i s c a l l has r e t u r n e d , t h e hardware CPUs c o r r e s p o n d i n g t o* < t t >cpus </ t t > may be s t o p p e d . S t o p p i n g t h e hardware CPUs e a r l i e r may* o t h e r w i s e r e s u l t i n d e a d l o c k s or l i f e l o c k s due t o s p i n l o o p s used f o r* i n t e r −t h r e a d communica t ion .** @param cpus t h e s e t o f CPUs t h a t s h o u l d be d i s a b l e d f o r use by t h e* JamaicaVM s c h e d u l e r .** @throws I l l e g a l A r g u m e n t E x c e p t i o n i f* < t t >! cpus . andNot ( a l l C p u s ( ) ) . i sEmpty ( ) < / t t > ( cpus c o n t a i n s a CPU t h a t i s n o t* i n < t t >a l l C p u s ( ) < / t t > , i . e , t h e c a l l e r a t t e m p t s t o d i s a b l e a non−e x i s t i n g* CPU) or i f < t t >enabledCpus ( ) . andNot ( cpus ) . i sEmpty ( ) < / t t > ( cpus i s a* s u p e r s e t o f e n a b l e d CPUs , i . e . , t h e c a l l e r a t t e m p t s t o d i s a b l e a l l CPUs ) .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" CpuContro l . d i s a b l e C p u s " ) </ t t >* d i s a b l i n g o f Cpus .* /

p u b l i c s t a t i c vo id d i s a b l e C p u s ( B i t S e t cpus ) ;

/ * ** e n a b l e t h e CPUs t h a t are s p e c i f i e d by t h e g i v e n B i t S e t .** A f t e r t h i s c a l l , < t t >enabledCpus ( ) < / t t > w i l l e q u a l o l d* < t t >enabledCpus ( ) . or ( cpus ) </ t t >.** Any t h r e a d s w i t h an { @link j a v a x . r e a l t i m e . A f f i n i t y } t h a t i s no l o n g e r* d i s j o i n t w i t h t h e new < t t >enabledCpus ( ) < / t t > w i l l aga in be a b l e t o run .** B e f o r e t h i s method i s c a l l e d , t h e hardware CPUs c o r r e s p o n d i n g t o


6 November 2014


* < t t >cpus </ t t > must have been s t a r t e d . Not s t a r t i n g t h e hardware CPUs* b e f o r e c a l l i n g t h i s method may o t h e r w i s e r e s u l t i n d e a d l o c k s or l i f e l o c k s* due t o s p i n l o o p s used f o r i n t e r −t h r e a d communica t ion .** @param cpus t h e s e t o f CPUs t h a t s h o u l d be e n a b l e d f o r use by t h e* JamaicaVM s c h e d u l e r .** @throws I l l e g a l A r g u m e n t E x c e p t i o n i f* < t t >! cpus . andNot ( a l l C p u s ( ) ) . i sEmpty ( ) < / t t > ( cpus c o n t a i n s a CPU t h a t i s n o t* i n < t t >a l l C p u s ( ) < / t t > , i . e , t h e c a l l e r a t t e m p t s t o e n a b l e a non−e x i s t i n g* CPU ) .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" CpuContro l . enab leCpus " ) </ t t >* e n a b l i n g o f Cpus .* /

p u b l i c s t a t i c vo id enab leCpus ( B i t S e t cpus ) ;

/ * ** Obta in t h e s e t o f f r e q u e n c i e s t h e g i v e n CPU can run a t .** @param cpuId t h e i d o f a CPU.** @return a s e t o f I n t e g e r s t h a t are p e r m i t t e d f r e q u e n c i e s g i v e n i n kHz o f* t h e s e l e c t e d CPU.* /

p u b l i c s t a t i c Set < I n t e g e r > a v a i l a b l e F r e q u e n c i e s ( i n t cpuId ) ;

/ * ** R e q u e s t t h e c u r r e n t f r e q u e n c y o f t h e g i v e n CPU** @param cpuId t h e i d o f a CPU.** @return t h e f r e q u e n c y i n kHz .* /

p u b l i c s t a t i c vo id c u r r e n t F r e q u e n c y ( i n t cpuId , i n t f r e q ) ;

/ * ** S e t t h e f r e q u e n c y o f t h e g i v e n CPU t o t h e g i v e n v a l u e** @param cpuId t h e i d o f a CPU.** @param t h e d e s i r e d f r e q u e n c y i n kHz .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" CpuContro l . d i s a b l e C p u s " ) </ t t >* d i s a b l i n g o f Cpus .* /

p u b l i c s t a t i c vo id s e t F r e q u e n c y ( i n t cpuId , i n t f r e q ) ;


Page 23


/ * ** R e q u e s t t h e c u r r e n t minimum f r e q u e n c y o f t h e g i v e n CPU. For a CPU whose* f r e q u e n c y i s changed d y n a m i c a l l y by t h e OS , t h i s d e f i n e s t h e lower bound o f* t h e p e r m i t t e d range o f f r e q u e n c y v a l u e s .** @param cpuId t h e i d o f a CPU.** @return t h e minimum f r e q u e n c y i n kHz .* /

p u b l i c s t a t i c i n t minFrequency ( i n t cpuId ) ;

/ * ** R e q u e s t t h e c u r r e n t maximum f r e q u e n c y o f t h e g i v e n CPU. For a CPU whose* f r e q u e n c y i s changed d y n a m i c a l l y by t h e OS , t h i s d e f i n e s t h e upper bound o f* t h e p e r m i t t e d range o f f r e q u e n c y v a l u e s .** @param cpuId t h e i d o f a CPU.** @return t h e minimum f r e q u e n c y i n kHz .* /

p u b l i c s t a t i c i n t maxFrequency ( i n t cpuId ) ;

/ * ** For a CPU whose f r e q u e n c y i s changed d y n a m i c a l l y by t h e OS , s e t t h e minimum* f r e q u e n c y o f t h e g i v e n CPU t o t h e g i v e n v a l u e** @param cpuId t h e i d o f a CPU.** @param t h e d e s i r e d minimum f r e q u e n c y i n kHz .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" CpuContro l . d i s a b l e C p u s " ) </ t t >* d i s a b l i n g o f Cpus .* /

p u b l i c s t a t i c vo id s e tMinFr equency ( i n t cpuId , i n t f r e q ) ;

/ * ** For a CPU whose f r e q u e n c y i s changed d y n a m i c a l l y by t h e OS , s e t t h e maximum* f r e q u e n c y o f t h e g i v e n CPU t o t h e g i v e n v a l u e** @param cpuId t h e i d o f a CPU.** @param t h e d e s i r e d maximum f r e q u e n c y i n kHz .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" CpuContro l . d i s a b l e C p u s " ) </ t t >* d i s a b l i n g o f Cpus .* /

p u b l i c s t a t i c vo id se tMaxFrequency ( i n t cpuId , i n t f r e q ) ;


6 November 2014


}

7.2 Freeze and Resume of Tasks

Live migration of tasks from one Java VM to another Java VM running on a different machine isa difficult task, in particular if the machines involved may use different hardware architecture ordifferent operating systems. The Java VM approach allows for an abstraction that nevertheless mayenable such a migration between very different underlying systems.

Migration support is much simpler if the threads that are to be migrated are in a defined state thatenables freezing of these threads and resuming without having to reconstruct the exact call contextand stack frames on the target systems. We therefore intend to provide APIs for different levels offreeze-and-resume support: Basic support that requires migrated threads to be stopped at a freezepoint, and full support that enables migration of threads at an arbitrary point during execution./ * *

* M i g r a t i o n p r o v i d e s VM− l e v e l s u p p o r t f o r m i g r a t i o n o f t a s k s from one Java VM* t o a n o t h e r Java VM.* /

p u b l i c c l a s s M i g r a t i o n{

/ *−−−−−−−−−−−−−−−−−−−−−−−−−−−− methods −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−* /

/ * ** F re ez e a Task t h a t i s i d e n t i f i e d by a s e t o f Threads r u n n i n g i n one* ThreadGroup , a s e t o f c l a s s e s lo ad ed by a g i v e n c l a s s l o a d e r and memory* a l l o c a t i o n t h r o u g h a g i v e n A l l o c a t i o n H a n d l e r .** F i r s t , t h e code w i l l w a i t f o r a l l t h r e a d s t o e x e c u t e a { @ f r e e z e P o i n t } . The* t h r e a d s w i l l be s t o p p e d a t t h e s e p o i n t s . Then , a l l t h e memory a l l o c a t e d* t h r o u g h t h e g i v e n a l l o c a t i o n hand le r , a l l t h e c l a s s e s lo ade d t h r o u g h t h e* g i v e n l o a d e r and a l l t h e t h r e a d s i n t h e g i v e n group w i l l be s e r i a l i z e d i n t o* an s t r e am o f b y t e s t h a t can be { @link resume } d on a n o t h e r v i r t u a l machine .** @param group a ThreadGroup t h a t s p e c i f i e s a l l t h e t h r e a d s o f t h i s t a s k t h a t* s h o u l d be f r o z e n .** @param load t h e Clas sLoader t h a t i d e n t i f i e s a l l t h e c l a s s e s t h a t b e lo ng t o* t h i s t a s k .** @param h a n d l e r t h e a l l o c a t i o n h a n d l e r t h a t i d e n t i f i e s a l l t h e o b j e c t s t h a t* be lo ng t o t h e t a s k .** @return a s t r e am o f b y t e s c o n t a i n i n g a l l t h e i n f o r m a t i o n on t h e f r o z e n t a s k .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" M i g r a t i o n . f r e e z e " ) </ t t >


Page 25


* f r e e z i n g o f t h r e a d s .* /

p u b l i c s t a t i c I n p u t S t r e a m f r e e z e ( ThreadGroup group ,C l a s s L o a d e r l o a d e r ,A l l o c a t i o n H a n d l e r h a n d l e r ) ;

/ * ** Resume a Task t h a t was s e r i a l i z e d by a p r e v i o u s c a l l t o { @link f r e e z e } .** A l l t h e o b j e c t s , c l a s s e s and t h r e a d s o f t h e f r o z e n t a s k w i l l be r e c r e a t e d* i n t h e c u r r e n t VM w i t h i n t h e framework o f t h e p r o v i d e d ThreadGroup ,* Clas sLoader and a l l o c a t i o n h a n d l e r . The t a s k s w i l l t h e n resume a t t h e* f r e e z e p o i n t s t h e y were s t o p p e d when f r e e z e ( ) was c a l l e d .** @param da ta t h e s t r ea m o f da ta produced by { @link f r e e z e } .** @param group t h e ThreadGroup t h e t h r e a d s w i l l be added t o .** @param load t h e Clas sLoader t h a t w i l l be used t o load t h e c l a s s e s o f t h e* t a s k .** @param h a n d l e r t h e a l l o c a t i o n h a n d l e r t h a t w i l l be used t o a l l o c a t e t h e* memory o f t h e t a s k .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" M i g r a t i o n . resume ") </ t t >* resuming o f t h r e a d s .* /

p u b l i c s t a t i c vo id resume ( I n p u t S t r e a m da ta ,ThreadGroup group ,C l a s s L o a d e r l o a d e r ,A l l o c a t i o n H a n d l e r h a n d l e r ) ;

/ * ** Pe rm i t f r e e z i n g o f t h e c u r r e n t t h r e a d f o r m i g r a t i o n . T h i s i s a no−op i n* case t h e t h r e a d i s n o t m i g r a t e d . However , i n case o f m i g r a t i o n , t h i s c a l l* w i l l n o t r e t u r n , b u t w i l l e x e c u t e < t t >r . run ( ) < / t t > when i t i s resumed . The* t h r e a d w i l l t e r m i n a t e once < t t >r . run ( ) < / t t > r e t u r n s .** At t h e p o i n t t h i s f u n c t i o n i s c a l l e d , t h e c u r r e n t t h r e a d must n o t have* e n t e r e d any Java m o n i t o r .** T h i s s h o u l d be used i n t h e < t t >run ( ) < / t t > methods o f a l l t h r e a d s t h a t* s h o u l d be p a r t o f a t a s k t h a t p e r m i t s m i g r a t i o n . An example use i s as* f o l l o w s :** <code >* c l a s s MyThread e x t e n d s Thread* {* p u b l i c v o i d run ( )* {


6 November 2014


* i n i t ( ) ;* e x e c u t e ( ) ;* }** p u b l i c v o i d e x e c u t e ( )* {* w h i l e ( t r u e )* {* / / p e r m i t m i g r a t i o n , on resume , c o n t i n u e w i t h e x e c u t e ( ) :* f r e z e e P o i n t ( ( ) −> e x e c u t e ( ) ) ;** A c t i o n a = g e t N e x t A c t i o n ( ) ;* a . per fo rm ( ) ;* }* }* }* </ code >** @param r t h e code t o e x e c u t e a f t e r a resume .** @throws I l l e g a l S t a t e E x c e p t i o n i f t h e c u r r e n t t h r e a d h o l d s any Java m o n i t o r .* /

p u b l i c s t a t i c vo id f r e e z e P o i n t ( Runnable r ) ;

/ * ** Wait f o r n o t i f i c a t i o n and p e r m i t f r e e z i n g o f t h e c u r r e n t t h r e a d f o r* m i g r a t i o n . T h i s i s e q u a l t o o . w a i t ( ) i n case t h e t h r e a d i s n o t m i g r a t e d .* However , i n case o f m i g r a t i o n , t h i s c a l l w i l l n o t r e t u r n , b u t w i l l e x e c u t e* < t t >r . run ( ) < / t t > when i t i s resumed . The t h r e a d w i l l t e r m i n a t e once* < t t >r . run ( ) < / t t > r e t u r n s .** At t h e p o i n t t h i s f u n c t i o n i s c a l l e d , t h e c u r r e n t t h r e a d must n o t have* e n t e r e d any Java m o n i t o r .** T h i s s h o u l d be used i n t h e < t t >run ( ) < / t t > methods o f a l l t h r e a d s t h a t* s h o u l d be p a r t o f a t a s k t h a t p e r m i t s m i g r a t i o n . An example use i s as* f o l l o w s :** <code >* c l a s s MyThread e x t e n d s Thread* {** L i s t <Ac t ion > a c t i o n L i s t ;** p u b l i c v o i d run ( )* {* i n i t ( ) ;* e x e c u t e ( ) ;* }** p u b l i c v o i d e x e c u t e ( )* {


Page 27


* w h i l e ( t r u e )* {* A c t i o n a c t i o n ;* s y n c h r o n i z e d ( a c t i o n L i s t )* {* w h i l e ( a c t i o n L i s t . i sEmpty ( ) )* {* / / p e r m i t m i g r a t i o n , on resume , c o n t i n u e w i t h e x e c u t e ( ) :* w a i t i n g F r e z e e P o i n t ( a c t i o n L i s t , ( ) −> e x e c u t e ( ) ) ;* }* a c t i o n = a c t i o n L i s t . removeHead ( ) ;* }** a c t i o n . per fo rm ( ) ;* }* }* }* </ code >** @param o t h e o b j e c t t h a t we are w a i t i n g f o r n o t i f i c a t i o n .** @param r t h e code t o e x e c u t e a f t e r a resume .** @throws I l l e g a l M o n i t o r S t a t e E x c e p t i o n i f t h e c u r r e n t t h r e a d does n o t own t h e* m o n i t o r a s s o c i a t e d w i t h o .** @throws I l l e g a l S t a t e E x c e p t i o n i f t h e c u r r e n t t h r e a d h o l d s any Java m o n i t o r* e x c e p t o .* /

p u b l i c s t a t i c vo id w a i t i n g F r e e z e P o i n t ( O b j e c t o , Runnable r ) ;

}

7.3 Freeze and Resume of whole VM

Alternatively to migrating single tasks identified by a ThreadGroup, a ClassLoader, and anAllocationHandler, the Java VM may provide means to migrate the whole state of the VMto another system. In this case, Java APIs are not sufficient since the current VM instance on theoriginal system will not survive the freezing and the whole VM state will be reconstructed on thetarget system.

Instead, a Java API may dump the VM state to a file, while the VM may permit a command lineoption to resume execution loading a state from a file. The freeze API would look as follows:

/ * ** F re ez e a l l t h r e a d s r u n n i n g on t h i s VM and s t o r e t h e s t a t e i n a new f i l e* w i t h t h e g i v e n name .** F i r s t , t h e code w i l l w a i t f o r a l l t h r e a d s t o e x e c u t e a { @ f r e e z e P o i n t } . The* t h r e a d s w i l l be s t o p p e d a t t h e s e p o i n t s . Then , a l l t h e memory a l l o c a t e d i n


6 November 2014


* t h e VM, a l l t h e c l a s s e s l oad ed and a l l t h e t h r e a d s w i l l be s e r i a l i z e d i n t o* an s t r e am o f b y t e s and saved i n t o a f i l e w i t h t h e g i v e n name such t h a t i t* can be resumed on a n o t h e r v i r t u a l machine .** T h i s c a l l n e v e r r e t u r n s . When resumed , t h e g i v e n Runnable w i l l be e x e c u t e d* by t h e resumed v e r s i o n o f t h e c u r r e n t t h r e a d on t h e t a r g e t s y s t e m .** @param f i l e N a m e t h e name o f t h e f i l e t o w r i t e t h e VM s t a t e t o .** @param r o p t i o n a l r u n n a b l e t o e x e c u t e a f t e r resume .** @throws S e c u r i t y E x c e p t i o n I f a s e c u r i t y manager has been i n s t a l l e d and i t* d e n i e s { @link R u n t i m e P e r m i s s i o n }< t t >(" M i g r a t i o n . f reezeVM ") </ t t >* f r e e z i n g o f t h e VM.* /

p u b l i c s t a t i c vo id f r e e z e ( S t r i n g f i leName ,Runnable r ) ;

To resume using the stored state, a VM command line option would permit reloading the state froma given file as follows:

> jamaicavm −c l a s s p a t h < c l a s s p a t h > −resume < f i l e n a m e >

After a resume, all threads will continue execution after the freeze point they were stopped in. Thethread that called freezewill continue execution in the Runnable instance provided as the secondargument to freeze.

7.4 Enhancing freeze and resume

In the general case, freezing and resuming of threads that are not stopped at a freezePoint willmake this approach much more flexible and usable. Threads that are not running are typically in oneof the following states:

• blocked in Java code waiting for notification, i.e. in method Object.wait().• blocked in Java code waiting for notification or a timeout, i.e., in methods such asObject.wait(long) or Thread.sleep().

• blocked waiting to enter a Java monitor owned by another thread• blocked in a system library native call, e.g., Socket.accept()• blocked in a application specific native call

If the virtual machine’s freeze and resume mechanism would be extended to transfer the executionstack of threads as well, these cases could be handled as well. The difficulties that will arise here areas follows:

• Code waiting for notification can be handled straightforward in a way very similar to codeexecuting a waitingFreezePoint.

• Code blocked with a timeout is similar, but we have to map the time on the source system to atime on the target.


Page 29


• Migration of threads blocked waiting to enter a monitor would mean that the internal state ofJava monitors would need to be transferred. Since this state typically is resolved soon (whenthe monitor becomes available again), it might be sufficient to wait for the thread to no longerbe in this state before freezing it.

• Code that is blocked in a system library native call requires special handling to resume the stateon the target depending on the kind of library call. The resume mechanism might try to restorethe exact state on the target, or, alternatively, return with an error from the native call if thismight make sense.

• Code that is blocked in a application specific native call cannot be supported by a general VMmigration approach so should be forbidden.

The approach to be taken by DreamCloud is to first gain experience with migration of different tasksand then to decide what additional support would be most useful.


6 November 2014


References

[1] aicas JamaicaVM – A hard realtime Java VM with a fully preemptable, deterministic garbagecollector. https://www.aicas.com/cms/en/JamaicaVM.

[2] aicas JamaicaVM API. http://www.aicas.com/jamaica/6.3/doc/jamaica_api/index.html.

[3] aicas JamaicaVM Realtime Specification for Java API. http://www.aicas.com/jamaica/6.3/doc/rtsj_api/index.html.

[4] Java Xlet. https://docs.oracle.com/javame/config/cdc/opt-pkgs/api/jsr927/javax/tv/xlet/Xlet.html.

[5] The JSR-282 Home Page: RTSJ version 2.0. http://www.jcp.org/en/jsr/detail?id=282.

[6] Manpage of cpufreq-set. https://www.kernel.org/pub/linux/utils/kernel/cpufreq/cpufreq-set.html.

[7] OpenJDK 8 open-source reference implementation of the Java SE 8 Platform Specificationdefined by JSR 337 in the Java Community Process. http://openjdk.java.net/projects/jdk8/.

[8] Osgi alliance. http://www.osgi.org/Specifications/HomePage.

[9] JUNIPER – Java Platform for High Performance and Realtime Large Scale Data Management,Project Number 318763, EC 7th framework programme. http://juniper-project.org, 2012–2015.

[10] Dominik Brodowski. [PATCH 2.5.56] cpufreq: sysfs interface. http://lwn.net/Articles/19798/, 2003.

[11] Tim Lindholm, Frank Yellin, Gilad Brache, and Alex Buckley. The Java Virtual Machine Spec-ification – Java SE 8 Edition. http://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf, 2014.

[12] T. Miettinen, D. Pakkala, and M. Hongisto. A Method for the Resource Monitoring of OSGi-based Software Components. In Software Engineering and Advanced Applications, SEAA ’08.34th Euromicro Conference, volume 5, pages 100–107, 2008.

[13] Institute of Electrical and Electronics Engineers. Portable operating system interface (posix),ieee std 1003.1. http://www.opengroup.org/onlinepubs/009695399, 2004.

[14] The DreamCloud Project. D1.2 – Dynamic Resource Allocation Requirements, 2014.

[15] Fridtjof Siebert. Concurrent, Parallel, Real-time Garbage-collection. SIGPLAN Not., 45(8):11–20, June 2010.


Page 31

https://www.aicas.com/cms/en/JamaicaVM

http://www.aicas.com/jamaica/6.3/doc/jamaica_api/index.html

http://www.aicas.com/jamaica/6.3/doc/jamaica_api/index.html

http://www.aicas.com/jamaica/6.3/doc/rtsj_api/index.html

http://www.aicas.com/jamaica/6.3/doc/rtsj_api/index.html

https://docs.oracle.com/javame/config/cdc/opt-pkgs/api/jsr927/javax/tv/xlet/Xlet.html

https://docs.oracle.com/javame/config/cdc/opt-pkgs/api/jsr927/javax/tv/xlet/Xlet.html

http://www.jcp.org/en/jsr/detail?id=282

http://www.jcp.org/en/jsr/detail?id=282

https://www.kernel.org/pub/linux/utils/kernel/cpufreq/cpufreq-set.html

https://www.kernel.org/pub/linux/utils/kernel/cpufreq/cpufreq-set.html

http://openjdk.java.net/projects/jdk8/

http://openjdk.java.net/projects/jdk8/

http://www.osgi.org/Specifications/HomePage

http://juniper-project.org

http://juniper-project.org

http://lwn.net/Articles/19798/

http://lwn.net/Articles/19798/

http://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf

http://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf

http://www.opengroup.org/onlinepubs/009695399

Documents

D4.1 – API Speciﬁcationapi.ning.com/files/8CzpakKglFRj9JYZrBB2sNfPkGGP-6D7U0xvhIpeaGaZQpn... · This deliverable describes the APIs that can be used to extend the existing functionality