WP 4 : Plugins and installation modules

www.eu-etics.org

INFSOM-RI-026753

WPWP44: : Plugins and Plugins and installation modulesinstallation modulesStatus ReportStatus Report

ETICS All-Hands – 23 May 2007ETICS All-Hands – 23 May 2007

PresenterPresenter: Eva Takács (4D SOFT): Eva Takács (4D SOFT)

INFSOM-RI-026753

Location of the plugins

• Location of the pluginsLocation of the plugins: : \ETICS_HOME\etics\lib\\ETICS_HOME\etics\lib\python2.3\site-packages\pluginspython2.3\site-packages\plugins directory of the directory of the etics client installation.etics client installation.

2ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

INFSOM-RI-026753

Location of plugins

• The The \ETICS_HOME\etics\lib\python2.3\site-packages\plugins\ETICS_HOME\etics\lib\python2.3\site-packages\plugins directory has additional subdirectories:directory has additional subdirectories:

• \\plugins_externalsplugins_externals – – containing the tools and their containing the tools and their dependenciesdependencies

• \plugins_buildfiles\plugins_buildfiles – containing the ant scripts with – containing the ant scripts with the core functionality of the implemented plugins. These ant the core functionality of the implemented plugins. These ant scripts are wrapped by the scripts are wrapped by the executeexecute()() function of the function of the PPluginluginManagerManager itself. itself.

• \plugins_properties\plugins_properties –containing the property files of –containing the property files of the ant scriptsthe ant scripts

• \plugins_docs\plugins_docs –containing the brief description of the –containing the brief description of the implemented pluginsimplemented plugins

INFSOM-RI-026753

Included tools as plugins

• At present the following tools are included as plugins:At present the following tools are included as plugins:• \cobertura\cobertura – containing the libraries of the java coverage – containing the libraries of the java coverage

tool coberturatool cobertura• \emma\emma – containing the libraries of the java coverage tool – containing the libraries of the java coverage tool

emmaemma• \junit\junit –containing junit library –containing junit library• \pmd\pmd – containing the libraries of the pmd tool – containing the libraries of the pmd tool• \findbugs\findbugs – containing the libraries of FindBugs – containing the libraries of FindBugs• \javaccn\javaccn – containing the libraries of javancss – containing the libraries of javancss• \jdepend\jdepend – containing the libraries of jdepend tool – containing the libraries of jdepend tool


INFSOM-RI-026753

Naming convention

• Naming convention of the pluginsNaming convention of the plugins: : JtoolnamePlugin.pyJtoolnamePlugin.py wherewhere

• ‘‘JJ’ – indicates that java programs are under analysis’ – indicates that java programs are under analysis• ‘‘toolnametoolname’ – refers to the integrated tool’ – refers to the integrated tool• ‘‘PluginPlugin’ – should be in the name of the module to ’ – should be in the name of the module to

enable the enable the PluginManager.pyPluginManager.py to load the plugin to load the plugin during the execution of the buildduring the execution of the build


INFSOM-RI-026753

JUnit plugin

• DefinitionDefinition:: this plugin is responsible for running JUnit this plugin is responsible for running JUnit test cases/suites using the widely-used ant test cases/suites using the widely-used ant mechanism.mechanism.

• Input directory:Input directory:• {workspaceDir}/repository/{projectName}/{workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch{moduleName}/{version}/noarch/lib /lib

• Output: xml, html reports located inOutput: xml, html reports located in• {workspaceDir}/reports/JUnitReports{workspaceDir}/reports/JUnitReports


INFSOM-RI-026753

Emma JUnit plugin

• Definition: Definition: this plugin is responsible for calculating this plugin is responsible for calculating EMMA coverage metrics for JUnit test cases.EMMA coverage metrics for JUnit test cases.


• Output junit results: xml, html reports located inOutput junit results: xml, html reports located in• {workspaceDir}/reports/JUnitReports{workspaceDir}/reports/JUnitReports-emma-emma

• Output emma coverage results: xml, html reports Output emma coverage results: xml, html reports located inlocated in

• {workspaceDir}/reports/{workspaceDir}/reports/coverage-emmacoverage-emma


INFSOM-RI-026753

Cobertura JUnit plugin

• Definition: Definition: this plugin is responsible for calculating this plugin is responsible for calculating coverage metrics for JUnit test casescoverage metrics for JUnit test cases using Cobertura using Cobertura..


• Output junit results: xml, html reports located inOutput junit results: xml, html reports located in• {workspaceDir}/reports/JUnitReports{workspaceDir}/reports/JUnitReports-cobertura-cobertura

• Output emma coverage results: xml, html reports Output emma coverage results: xml, html reports located inlocated in

• {workspaceDir}/reports/{workspaceDir}/reports/coverage-coberturacoverage-cobertura


INFSOM-RI-026753

JDepend plugin

• Definition: this plugin is responsible for scanning Java byte code and generating dependency info. The open source jdepend tool is used in this plugin.

• Input directory: {workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/lib

• Output:• Html(jdepend-report.html) and xml(jdepend-report.xml) reports are created in {workspaceDir}/reports directory.

INFSOM-RI-026753

JPmd plugin

Definition: this plugin is responsible for scanning Java source code and looking for potential problems like:

• Possible bugs - empty try/catch/finally/switch statements

• Dead code - unused local variables, parameters and private methods

• Suboptimal code - wasteful String/StringBuffer usage• Overcomplicated expressions - unnecessary if

statements, for loops that could be while loops

The open source pmd tool is used in this plugin.

INFSOM-RI-026753

JPmd plugin

• Input directory: {workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/src

• Output:• Html (pmdReport.html) and xml (pmdReport.xml)

reports are created in {workspaceDir}/reports directory.

INFSOM-RI-026753

Interpreting the results

• PMD works by scanning Java code and checks for violations in three major areas:


INFSOM-RI-026753


• Compliance with coding standards such as:

– Naming conventions - class, method, parameter and variable names

– Class and method length – Existence and formatting of comments and JavaDocs

INFSOM-RI-026753


• Coding antipatterns such as:

– Empty try/catch/finally/switch blocks – Unused local variables, parameters and private methods– Empty if/while statements – Overcomplicated expressions - unnecessary if statements, for

loops that could be while loops – Classes with high Cyclomatic Complexity measurements

INFSOM-RI-026753


• More precisely, PMD defines rulesets and inside each ruleset a set of rules in order to verify the source code.

• The default of rulesets are as follows:

• # Basic (rulesets/basic.xml) – looks for some basic problems, for example, catch blocks shouldn't be empty, override hashCode() anytime you override equals(), etc.

• # Naming (rulesets/naming.xml) -- Tests for the standard Java naming conventions: variable names should not be too short; method names should not be too long; class names should begin with an uppercase letter, method and field names should begin with a lowercase letter, etc.

INFSOM-RI-026753


• # Import statements (rulesets/imports.xml) -- Checks for minor issues with import statements, such as importing the same class twice or importing a class from java.lang.

• # JUnit tests (rulesets/junit.xml) -- Looks for specific issues with test cases and test methods, such as correct spelling of method names and whether suite() methods are static and public.

INFSOM-RI-026753


• # Strings (rulesets/string.xml) -- Identifies common problems that when working with strings, such as duplicate string literals, calling the String constructor, and calling toString() on String objects.

• • # Braces (rulesets/braces.xml) -- Checks whether for, if, while, and

else statements use braces.• • # Code size (rulesets/codesize.xml) -- Tests for overly long

methods, classes with too many methods, and similar candidates for refactoring.

• • # Javabeans (rulesets/javabeans.xml) -- Inspects JavaBeans

components for violations of JavaBeans coding conventions, such as unserializable bean classes.

INFSOM-RI-026753

Interpreting the result

• # Strict exceptions (rulesets/strictexception.xml) -- A few more tests for exceptions: methods should not be declared to throw java.lang.Exception, exceptions should not be used for flow control, Throwable should not be caught.

• # Controversial (rulesets/controversial.xml) -- This rule set contains some of the more questionable checks, including assigning null to a variable, multiple return points from a method, and importing from the sun packages.

• # Logging (rulesets/logging-java.xml) -- Searches for strange uses

of java.util.logging.Logger, including nonfinal, nonstatic loggers, and more than one logger in a class.

INFSOM-RI-026753

JFindBugs plugin

• Definition: this plugin is responsible for scanning Java byte code and looking for potential bugs. The open source findbugs tool is used in this plugin.

• Input directory: {workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/lib

• Output:

Html (findbugsReport.html) and xml (findbugsReport.xml) reports are created in {workspaceDir}/reports directory.

INFSOM-RI-026753

Interpreting results

• FindBugs results are stored in html/xml reports.• The html report contains a summary of the bug

categories indicating their severity.• Then each bug is detailed and the location of it in the

source file is also specified. • Lastly, an explanation of the bugs found are put at the

end of the report. One example of such a report can be found

INFSOM-RI-026753

Interpreting results

• FindBugs looks for potential problems by matching the bytecodes against a list of bug patterns.

• Bug patterns are code idioms that are often errors. • FindBugs doesn't focus on style or formatting; it

specifically tries to find real bugs or potential performance problems.

• Bug patterns in principally fall in the following categories:– Single-threaded correctness issue– Thread/synchronization correctness issue– Performance issue– Security and vulnerability to malicious untrusted code

INFSOM-RI-026753

OS - Open Stream

• When a program opens an input or output stream, it is good practice to ensure that the stream is closed when it becomes unreachable. Although finalizers ensure that Java I/O streams are automatically closed when they are garbage collected, there is no guarantee that this will happen in a

• timely manner. There are two reasons why streams should be closed as early as possible. First, operating system file descriptors are a limited resource, and running out of them may cause the program to misbehave. Another reason is that if a buffered output stream is not closed, the data stored

• in the stream’s buffer may never be written to the file (because Java finalizers are not guaranteed to be run when the program exits).

•

INFSOM-RI-026753

OS - Open Stream

• An example of an open stream found by the bug detector is as follows:

• • private static File _parsePackagesFile(File packages, File destDir) {

• try {• FileReader fr =new FileReader(packages);

• BufferedReader br =new BufferedReader(fr);

• ...• // fr/br are never closed• }

INFSOM-RI-026753

NP – Null pointer dereference

• NP – Null pointer dereference• Calling a method or accessing an instance field of a null value

results in a NullPointerException at runtime. This detector looks for instructions where a null reference might be dereferenced.

• • An example of a null pointer dereference found by the NP bug

detector is as follows:• if (entry == null) {• IClasspathContainer

container=JavaCore.getClasspathContainer (• entry.getPath(), // entry is null!• root.getJavaProject());• .....• }

INFSOM-RI-026753

UR - Uninitialized Read In Constructor

• When a new object is constructed, each field is set to the default value for its type. In general, it is not useful to read a field of an object before a value is written to it. This detector finds fields that are read in constructors before they're initialized.

• • public class Thing {• private List actions;• public Thing(String startingActions) {• StringTokenizer tokenizer = new

StringTokenizer(startingActions);• while (tokenizer.hasMoreTokens()) {• actions.add(tokenizer.nextToken());• } • }• }• In this example, line red will cause a null pointer exception because the

variable actions has not been initialized.

INFSOM-RI-026753

Installation modules

• The aim of installation modules is to assist users in their installation procedure by providing some general purpose installation modules, for example, for tomcat, mysql, wscore

INFSOM-RI-026753

Installation modules

• In order to be more flexible, it was intended to implement a general installation procedure for all software packages while the individual features such as the commands for testing the installation should be included in a separate config file.

INFSOM-RI-026753

Steps of installation

• To have (download) the requested package• The implemented automatism guess the file type from

the mime type and unpacks or installs the packages accordingly

• Then the package is installed in the specified location• The last step is to check the installation using the

testing command specified in the config file. By default the testing command is provided, but the user can change it.

INFSOM-RI-026753

Config file

• [TOMCAT]• tomcat.url =

http://xenia.sote.hu/ftp/mirrors/www.apache.org/tomcat/tomcat-6/v6.0.10/bin/apache-tomcat-6.0.10-deployer.tar.gz

• tomcat.destdir = /home/takacs/Applications/tomcat• tomcat.installcmd = mkdir -p %(tomcat.destdir)s && rm -

fr %(tomcat.destdir)s/* && cp -r %(tomcat.unpackpath)s/* %(tomcat.destdir)s/

• tomcat.javahome = $(dirname $(dirname $(which java)))• tomcat.startcmd = (JAVA_HOME=%(tomcat.javahome)s %

(tomcat.destdir)s/apache-tomcat-6.0.10/bin/startup.sh)• tomcat.checkcmd = (sleep 21 && wget

http://localhost:8080 -O /tmp/TOMCAT.TEST && cat /tmp/TOMCAT.TEST)

• tomcat.stopcmd = (JAVA_HOME=%(tomcat.javahome)s %(tomcat.destdir)s/apache-tomcat-6.0.10/bin/shutdown.sh)

• tomcat.uninstallcmd = echo -e '\ntomcat uninstall command\n'

•

INFSOM-RI-026753

Etics

• 1. etics-show-module-structure externals | grep tomcat• • Result:• >Component : tomcat• • 2. etics-checkout --project-config externals tomcat• • Result:• >Downloading configuration 'tomcat v. 5.0.28' from • >repository

'http://eticssoft.web.cern.ch/eticssoft/repository//externals/tomcat/5.0.28/noarch/tomcat5->5.0.28.tar.gz'

• • At this point ’tomcat5-5.0.28.tar.gz’ is located in

{workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch.

INFSOM-RI-026753

YUM

• If the ETICS repository does not contain the required tool, then the next option is to use the YUM mechanism. In order to successfully use YUM the /etc/yum.repos.d/*.repo files should be present and filled with the necessary data.

• which yum• yum update (updating just metadata not libraries)• yum list all | grep tomcat • yum install tomcat

INFSOM-RI-026753

Apt

• Lastly, if no YUM mechanism is provided then apt is used. The following commands are run:

• which apt• apt-search tomcat (the biggest version of the

component is searched)• apt-get update (updating metadata)• apt-get upgrade (upgrading the libraries)• apt-get install

Documents

WP 4 : Plugins and installation modules