Course Empirical Software Engineeringlaemmel/esecourse/...© 2013 101companies and Ralf Lämmel...

Metadata inferenceCourse "Empirical Software Engineering"

University of Koblenz-Landau

Department of Computer Science

Ralf Lämmel

Software Languages Team

Overall research context

Objective

https://commons.wikimedia.org/wiki/File:Prism-rainbow.svg

Your arbitrary program

Languages

Technologies

Concepts

Features

Raw research questions

How to automatically detect

used software languages,

used software technologies,

relevant software concepts,

implemented system features?

... perhaps specifically in the context of the 101companies project

Too many languages.

Too many technologies.

Too many concepts.

Too little time.

SQL DDL

XLSTSaxon

Hibernate

RestOWL

SparQLXSLT

Prolog

jDOMRose

Protegé

XQuery

XMLSpy

MySQLArgoUML

Dragan

EMF.gen

ORACLE

Jersey

Sesame

Stratego

JeanBeans

RDF(S)

SLE2010xerces

saxonsax

JMI JMF

What’s the problem?

Research challengesNew 101contributions with unforeseen technologies and concepts and new ways of implementing features.

Different platforms and repositories: Apache, .NET Framework, gem, Maven repository, hackageDB, ...

Lack of an established ontology: sets of tags or categories used by the different platforms and repositories.

Overall: scalability, generality6

Let’s focus on software chrestomathies.

Software chrestomathies

What’s a software chrestomathy?

• Quick definition

‣ A collection of software systems (‘programs’)✦ designed as an aid in learning a CS subject ✦ exercising diverse languages & technologies

• Examples (more or less)

‣ 99 Bottles of Beer

‣ Rosetta Code

‣ HollingBerries

‣ 101companies

Company X:Swing + JDBC

Company Y:SWT + Hibernate

Company Z:GWT + MongoDB

The 101companies chrestomathyA collection of human-resources management systems:

code + documentation

• Total salaries

• Increase salaries

• Cut salaries

• Edit employee data

• Import / export company data

“101” covers many languages• Java

• C#

• VB.NET

• Python

• XML

• GIF

• Haskell

• XHTML

• .properties

• .ini

• HTML

• JSON

• Ecore

• PHP

• PHP4

• PHP5

• SQL

• .sh

• PNG

• ATL

• XSD

• Scala

• Text

• Markdown

• Prolog

• F#

• Perl

• CSS

• XMI

• AspectJ

• Smalltalk

• JAR

• Erlang

• WSDL

• Rascal

• Javascript

• 101meta

• Ruby

• C++

• Cobol

• (make)

• (Ant)

• ...

• make

• Ant

• Maven

• sbt

• JAXB

• xjc

• EF

• xsd.exe

• XmlSerializer

• ANTLR

• javac

• Eclipse

• Hibernate

• Java collections

• VS

• java.util

• java.io

• java.lang.reflect

• LINQ

• System.Xml

• System.Xml.Linq

• ADO

• SOA

• JPA

• EMFCompare

• 101explorer

• Mediawiki

• hackage

• cabal

• ghci

• ghc

• Swing

• AWT

• Java RMI

• JDBC

• zip

• jEdit

• jdom

• w3c.dom

• dom4j

• xom

• SAX

• ...

“101” covers many technologies

One particular implementation

Part of the documentation

Traceabilitybetween code and documentation

The traceability challengefor a general software product

The traceability challengefor a software chrestomathy

The traceability challenge

Metamodels at hand101repo vs. 101wiki

The 101meta language for rule-based metadata specification

• Check condition on files

• Associate metadata with files or fragments

‣ Languages

‣ Technologies

‣ Concepts

‣ Tools for validation, fragment selection, ...

“Metadata inference”

Links established by 101meta rules

101meta rules for ANTLR

Wanted!101meta rules for ANTLR

When is a file ...

• ... the ANTLR library?

• ... an ANTLR grammar?

• ... an ANTLR-generated parser?

• ... a source that imports ANTLR?

(See more rules online.)

What’s ANTLR?company : 'company' STRING '{' department* '}' EOF; department : 'department' STRING '{' ('manager' employee) ('employee' employee)* department* '}';

employee : STRING '{' 'address' STRING 'salary' FLOAT '}';

ANTLR style grammar for companies

Grammars were hot in the 70ies?You are so wrong!

http://www.antlr.org

[−] ANTLR theme ANTLR-centric grammarware theme

yapg A ANTLR-based generator for text-to-object mappings

antlrAcceptor An ANTLR-based acceptor for textual syntax

antlrLexer Lexer-based processing with ANTLR

antlrObjects Object/Text mapping for Java with ANTLR for parsing

antlrParser Processing textual syntax with semantic actions of ANTLR

antlrTrees Parsing text to trees and walk them with ANTLR

gra2mol Grammar to model transformation with Grammar2Model

xtext An XText- and Eclipse-based DSL editor

http://101companies.org/wiki/Theme:ANTLR

{ "basename" : "#^antlr-(.*)\\.jar$#", "metadata" : { "partOf" : "ANTLR", "comment" : "The ANTLR library, Version $1" } }

“Condition” of the rule

Assigned metadata

Expressed relationship

A 101meta rule to find the ANTLR library

{ "suffix" : ".g", "metadata" : { "inputOf" : "ANTLR", "comment" : "An ANTLR grammar" } }

A 101meta rule to find an ANTLR grammar

“Condition” of the rule

Assigned metadata

Expressed relationship

{ "basename" : "#^.*Parser\\.java$#", "content" : "// \\$ANTLR.*\\.g", "metadata" : [ { "outputOf" : "ANTLR", "comment" : "An ANTLR-generated parser" }, { "concept" : "Parser" } ] }

Condition on text of file

Metadata for software concept

A 101meta rule to find a generated parser

{ "suffix" : ".java", "predicate" :

"technologies/Java_platform/javaImport.sh", "args" : [ "org.antlr.runtime" ], "metadata" : { "dependsOn" : "ANTLR", "comment" : "A source that imports ANTLR" } }

Programmatic condition

Java package to be tested for

A rule to find a source importing ANTLR

Metadata management

• Metadata declaration

with rules, as demonstrated

• Metadata assignment

with a rule engine

• Metadata exploration

with a browser (as seen in second)

Metadata assignment101worker continuously walks

over 101repo to construct data

for files matches 101meta

rules. In fact, each file is

associated with such matches.

http://data.101companies.org/resources/contributions/antlrObjects/org/softlang/parser/CompanyParser.java.summary.json

Metadata exploration

Disclaimer: This is not even beta.

Metadata exploration

Disclaimer: This is not even beta.

Platforms

HackageDBhttp://hackage.haskell.org

The Apache Software Foundationhttp://projects.apache.org/

What’s that?

PyPi - the Python Package Indexhttps://pypi.python.org/

Maven Central Repositoryhttp://search.maven.org/

What’s that?

http://rubygems.org/

Microsoft Developer Networkhttp://msdn.microsoft.com/

Thanks!Questions?

Course Empirical Software Engineeringlaemmel/esecourse/...© 2013 101companies and Ralf Lämmel...

Documents

Neural Networks and Data Mining Folie 1 Artificial Neural Networks and Data Mining Uwe Lämmel Wismar Business School laemmel Uwe.Laemmel@hs-wismar.de

Programming Languages Seminar, Spring 20041 Grammar Adaptation (Ralf Lämmel, CWI) Presentation and slides by: Faizan Javed March 9 th, 2004

Einführung in die Programmierung Folie 1 Uwe Lämmel Einführung in die Programmierung Zusammenfassung Uwe Lämmel Fakultät für Wirtschaftswissenschaften

Introduction - Uni Koblenz-Landaulaemmel/esecourse/slides/intro.pdf · This presentation is available free for non-commercial use with attribution under a creative commons license

Exploratory research on API usage - Uni Koblenz-Landaulaemmel/esecourse/... · 2014. 7. 24. · 2]; the wrapper must neutralize these diﬀerences. Our example concerns two GUI APIs:

Wissenschaftliches Arbeiten - Auswertung Folie 1 Artificial Neural Networks Uwe Lämmel Wismar Business School laemmel Uwe.Laemmel@hs-wismar.de

Aspect-oriented Programming - Uni Koblenz-Landaulaemmel/pttcourse/slides/aop.pdf · (C) 2010-2013 Prof. Dr. Ralf Lämmel, Universität Koblenz-Landau (where applicable) Logistics

Programmierung mit XML - userpageslaemmel/oopm/slides/xml.pdf · (C) Ralf Lämmel, OOPM, Universität Koblenz-Landau Programmierung mit XML XML als Austausch- und Speicherformat

An Empirical Assessment of Empirical Corporate Finance

Anweisungen und Datentypen Folie 1 Uwe Lämmel Einführung in die Programmierung Anweisungen und Datentypen Uwe Lämmel Fakultät für Wirtschaftswissenschaften

Web application development - Uni Koblenz-Landaulaemmel/pttcourse/slides/web.pdfWeb application development Software Languages Team! University of Koblenz-Landau! Ralf Lämmel and

Einführung in die Programmierung – Sammlungen Folie 1 Uwe Lämmel Einführung in die Programmierung Datensammlung Uwe Lämmel Fakultät für Wirtschaftswissenschaften

Mechanistic-Empirical Empirical Pavement Design Using

Empirical Formulas & Molecular Formulas Empirical Formulas Molecular Formulas

Empirical Study of Credit Rating Migration in Indiamanagement-review.nmims.edu/.../empirical-study-of-credit...ghosh.pdf · Empirical Study of Credit Rating Migration in India Empirical

Stefan Hanenberg (University of Duisburg-Essen)laemmel/esecourse/slides/ctrlexp2.pdf · Grundlage Signifikanztests Signifikanztests ergeben p-Wert, der die Größe des alpha-Fehlers

Empirical Formula and Molecular Formula. Empirical Formula Empirical formula is the simplest whole…

Ralf Lämmel Software Languages Team University of Koblenz ...laemmel/pttcourse/slides/waf.pdf · © 2014-2015, Ralf Lämmel, rlaemmel@acm.org Visited 9 July 2015

Scrap your boilerplate: generic programming in Haskell Ralf Lämmel, Vrije University Simon Peyton Jones, Microsoft Research

Ralf Lämmel Software Languages Team University of Koblenz ...laemmel/pttcourse/slides/mappin… · • Model-group definitions • Simple type definitions • Attribute declarations