title: An exploration of program as language

authors:
Elisa L. A. Baniassad
Clayton G. Myers

abstract: In this paper we explore the idea that the code that constitutes a program actually forms a higher-level, program specific language. The symbols of the language are the abstractions of the program, and the grammar of the language is the set of (generally unwritten) rules about the allowable combinations of those abstractions. As such, a program is both a language definition, and the only use of that language. This specificity means that reading a never-before encountered program involves learning a new natural language, and that porting code from one program to another requires translation from one natural language into another. We suggest that the complexity and depth of the program language is affected by the gap between the program semantics (what the program is meant to do) and the code semantics (the way in which the machine runs). We believe that in seeing that programs are languages, we gain new insight into our own experience as programmers, and are able to gain new perspective on the intense complexity of code and its creation.


venue: oopsla

title: Grace: safe multithreaded programming for C/C++

authors:
Emery D. Berger
Ting Yang
Tongping Liu
Gene Novark

abstract: The shift from single to multiple core architectures means that programmers must write concurrent, multithreaded programs in order to increase application performance. Unfortunately, multithreaded applications are susceptible to numerous errors, including deadlocks, race conditions, atomicity violations, and order violations. These errors are notoriously difficult for programmers to debug.


title: Thorn: robust, concurrent, extensible scripting on the JVM

authors:
Bard Bloom
John Field
Nathaniel Nystrom
Johan Östlund
Gregor Richards
Rok Strnisa
Jan Vitek
Tobias Wrigstad

abstract: Scripting languages enjoy great popularity due to their support for rapid and exploratory development. They typically have lightweight syntax, weak data privacy, dynamic typing, powerful aggregate data types, and allow execution of the completed parts of incomplete programs. The price of these features comes later in the software life cycle. Scripts are hard to evolve and compose, and often slow. An additional weakness of most scripting languages is lack of support for concurrency - though concurrency is required for scalability and interacting with remote services. This paper reports on the design and implementation of Thorn, a novel programming language targeting the JVM. Our principal contributions are a careful selection of features that support the evolution of scripts into industrial grade programs - e.g., an expressive module system, an optional type annotation facility for declarations, and support for concurrency based on message passing between lightweight, isolated processes. On the implementation side, Thorn has been designed to accommodate the evolution of the language itself through a compiler plugin mechanism and target the Java virtual machine.


title: A type and effect system for deterministic parallel Java

authors:
Robert L. Bocchino Jr.
Vikram S. Adve
Danny Dig
Sarita V. Adve
Stephen Heumann
Rakesh Komuravelli
Jeffrey Overbey
Patrick Simmons
Hyojin Sung
Mohsen Vakilian

abstract: Today's shared-memory parallel programming models are complex and error-prone.While many parallel programs are intended to be deterministic, unanticipated thread interleavings can lead to subtle bugs and nondeterministic semantics. In this paper, we demonstrate that a practical type and effect system can simplify parallel programming by guaranteeing deterministic semantics with modular, compile-time type checking even in a rich, concurrent object-oriented language such as Java. We describe an object-oriented type and effect system that provides several new capabilities over previous systems for expressing deterministic parallel algorithms.We also describe a language called Deterministic Parallel Java (DPJ) that incorporates the new type system features, and we show that a core subset of DPJ is sound. We describe an experimental validation showing thatDPJ can express a wide range of realistic parallel programs; that the new type system features are useful for such programs; and that the parallel programs exhibit good performance gains (coming close to or beating equivalent, nondeterministic multithreaded programs where those are available).


title: Strictly declarative specification of sophisticated points-to analyses

authors:
Martin Bravenboer
Yannis Smaragdakis

abstract: We present the DOOP framework for points-to analysis of Java programs. DOOP builds on the idea of specifying pointer analysis algorithms declaratively, using Datalog: a logic-based language for defining (recursive) relations. We carry the declarative approach further than past work by describing the full end-to-end analysis in Datalog and optimizing aggressively using a novel technique specifically targeting highly recursive Datalog programs.


title: Accelerating the creation of customized, language-Specific IDEs in Eclipse

authors:
Philippe Charles
Robert M. Fuhrer
Stanley M. Sutton Jr.
Evelyn Duesterwald
Jurgen J. Vinju

abstract: Full-featured integrated development environments have become critical to the adoption of new programming languages. Key to the success of these IDEs is the provision of services tailored to the languages. However, modern IDEs are large and complex, and the cost of constructing one from scratch can be prohibitive. Generators that work from language specifications reduce costs but produce environments that do not fully reflect distinctive language characteristics.


title: On understanding data abstraction, revisited

abstract: In 1985 Luca Cardelli and Peter Wegner, my advisor, published an ACM Computing Surveys paper called "On understanding types, data abstraction, and polymorphism". Their work kicked off a flood of research on semantics and type theory for object-oriented programming, which continues to this day. Despite 25 years of research, there is still widespread confusion about the two forms of data abstraction, abstract data types and objects. This essay attempts to explain the differences and also why the differences matter.


title: Optimizing programs with intended semantics

authors:
Daniel von Dincklage
Amer Diwan

abstract: Modern object-oriented languages have complex features that cause programmers to overspecify their programs. This overspecification hinders automatic optimizers, since they must preserve the overspecified semantics. If an optimizer knew which semantics the programmer intended, it could do a better job.


title: Empirical assessment of object-oriented implementations with multiple inheritance and static typing

authors:
Roland Ducournau
Floréal Morandat
Jean Privat

abstract: Object-oriented languages involve a threefold tradeoff between runtime efficiency, expressiveness (multiple inheritance), and modularity, i.e. open-world assumption (OWA). Runtime efficiency is conditioned by both the implementation technique and compilation scheme. The former specifies the data structures that support method invocation, attribute access and subtype testing. The latter consists of the production line of an executable from the source code. Many implementation techniques have been proposed and several compilation schemes can be considered from fully global compilation under the closed-world assumption (CWA) to separate compilation with dynamic loading under the OWA, with midway solutions. This article reviews a significant subset of possible combinations and presents a systematic, empirical comparison of their respective efficiencies with all other things being equal. The testbed consists of the Prm compiler that has been designed for this purpose. The considered techniques include C++ subobjects, coloring, perfect hashing, binary tree dispatch and caching. A variety of processors were considered. Qualitatively, these first results confirm the intuitive or theoretical abstract assessments of the tested approaches. As expected, efficiency increases as CWA strengthens. From a quantitative standpoint, the results are the first to precisely compare the efficiency of techniques that are closely associated with specific languages like C++ and Eiffel. They also confirm that perfect hashing should be considered for implementing Java and .Net interfaces.


title: Profile-guided static typing for dynamic scripting languages

authors:
Michael Furr
Jong-hoon (David) An
Jeffrey S. Foster

abstract: Many popular scripting languages such as Ruby, Python, and Perl include highly dynamic language constructs, such as an eval method that evaluates a string as program text. While these constructs allow terse and expressive code, they have traditionally obstructed static analysis. In this paper we present PRuby, an extension to Diamondback Ruby (DRuby), a static type inference system for Ruby. PRuby augments DRuby with a novel dynamic analysis and transformation that allows us to precisely type uses of highly dynamic constructs. PRuby's analysis proceeds in three steps. First, we use run-time instrumentation to gather per-application profiles of dynamic feature usage. Next, we replace dynamic features with statically analyzable alternatives based on the profile. We also add instrumentation to safely handle cases when subsequent runs do not match the profile. Finally, we run DRuby's static type inference on the transformed code to enforce type safety.


title: A concurrent dynamic analysis framework for multicore hardware

authors:
Jungwoo Ha
Matthew Arnold
Stephen M. Blackburn
Kathryn S. McKinley

abstract: Software has spent the bounty of Moore's law by solving harder problems and exploiting abstractions, such as high-level languages, virtual machine technology, binary rewriting, and dynamic analysis. Abstractions make programmers more productive and programs more portable, but usually slow them down. Since Moore's law is now delivering multiple cores instead of faster processors, future systems must either bear a relatively higher cost for abstractions or use some cores to help tolerate abstraction costs.


title: How a Java VM can get more from a hardware performance monitor

authors:
Hiroshi Inoue
Toshio Nakatani

abstract: This paper describes our sampling-based profiler that exploits a processor's HPM (Hardware Performance Monitor) to collect information on running Java applications for use by the Java VM. Our profiler provides two novel features: Java-level event profiling and lightweight context-sensitive event profiling. For Java events, we propose new techniques to leverage the sampling facility of the HPM to generate object creation profiles and lock activity profiles. The HPM sampling is the key to achieve a smaller overhead compared to profilers that do not rely on hardware helps. To sample the object creations with the HPM, which can only sample hardware events such as executed instructions or cache misses, we correlate the object creations with the store instructions for Java object headers. For the lock activity profile, we introduce an instrumentation-based technique, called ProbeNOP, which uses a special NOP instruction whose executions are counted by the HPM. For the context-sensitive event profiling, we propose a new technique called CallerChaining, which detects the calling context of HPM events based on the call stack depth (the value of the stack frame pointer). We show that it can detect the calling contexts in many programs including a large commercial application. Our proposed techniques enable both programmers and runtime systems to get more valuable information from the HPM to understand and optimize the programs without adding significant runtime overhead.


title: Providing rapid feedback in generated modular language environments: adding error recovery to scannerless generalized-LR parsing

authors:
Lennart C. L. Kats
Maartje de Jonge
Emma Nilsson-Nyman
Eelco Visser

abstract: Integrated development environments (IDEs) increase programmer productivity, providing rapid, interactive feedback based on the syntax and semantics of a language. A heavy burden lies on developers of new languages to provide adequate IDE support. Code generation techniques provide a viable, efficient approach to semi-automatically produce IDE plugins. Key components for the realization of plugins are the language's grammar and parser. For embedded languages and language extensions, constituent IDE plugin modules and their grammars can be combined. Unlike conventional parsing algorithms, scannerless generalized-LR parsing supports the full set of context-free grammars, which is closed under composition, and hence can parse language embeddings and extensions composed from separate grammar modules. To apply this algorithm in an interactive environment, this paper introduces a novel error recovery mechanism, which allows it to be used with files with syntax errors -- common in interactive editing. Error recovery is vital for providing rapid feedback in case of syntax errors, as most IDE services depend on the parser from syntax highlighting to semantic analysis and cross-referencing. We base our approach on the principles of island grammars, and derive permissive grammars with error recovery productions from normal SDF grammars. To cope with the added complexity of these grammars, we adapt the parser to support backtracking. We evaluate the recovery quality and performance of our approach using a set of composed languages, based on Java and Stratego.


title: pi: a pattern language

authors:
Roman Knöll
Mira Mezini

abstract: Current programming languages and techniques realize many features which allow their users to extend these languages on a semantic basis: classes, functions, interfaces, aspects and other entities can be defined. However, there is a lack of modern programming languages which are both semantically and syntactically extensible from within the language itself, i.e., with no additional tool or meta-language. In this paper we present π as an approach that aims to overcome this lack. π provides an abstraction mechanism based on parameterized symbols which is capable of semantically and syntactically unifying programming concepts like variables, control-structures, procedures and functions into one concept: the pattern. We have evaluated the abstraction potential and the syntactic extensibility of π by successfully creating patterns for the aforementioned programming concepts. π could serve as a tool for designing new experimental languages and might generally influence the view we have on current programming concepts.


title: Debug all your code: portable mixed-environment debugging

authors:
Byeongcheol Lee
Martin Hirzel
Robert Grimm
Kathryn S. McKinley

abstract: Programmers build large-scale systems with multiple languages to reuse legacy code and leverage languages best suited to their problems. For instance, the same program may use Java for ease-of-programming and C to interface with the operating system. These programs pose significant debugging challenges, because programmers need to understand and control code across languages, which may execute in different environments. Unfortunately, traditional multilingual debuggers require a single execution environment.


title: The design of a task parallel library

authors:
Daan Leijen
Wolfram Schulte
Sebastian Burckhardt

abstract: The Task Parallel Library (TPL) is a library for .NET that makes it easy to take advantage of potential parallelism in a program. The library relies heavily on generics and delegate expressions to provide custom control structures expressing structured parallelism such as map-reduce in user programs. The library implementation is built around the notion of a task as a finite CPU-bound computation. To capture the ubiquitous apply-to-all pattern the library also introduces the novel concept of a replicable task. Tasks and replicable tasks are assigned to threads using work stealing techniques, but unlike traditional implementations based on the THE protocol, the library uses a novel data structure called a 'duplicating queue'. A surprising feature of duplicating queues is that they have sequentially inconsistent behavior on architectures with weak memory models, but capture this non-determinism in a benign way by sometimes duplicating elements. TPL ships as part of the Microsoft Parallel Extensions for the .NET framework 4.0, and forms the foundation of Parallel LINQ queries (however, note that the productized TPL library may differ in significant ways from the basic design described in this article).


title: Parallel programming with object assemblies

authors:
Roberto Lublinerman
Swarat Chaudhuri
Pavol Cerný

abstract: We present Chorus, a high-level parallel programming model suitable for irregular, heap-manipulating applications like mesh refinement and epidemic simulations, and JChorus, an implementation of the model on top of Java. One goal of Chorus is to express the dynamic and instance-dependent patterns of memory access that are common in typical irregular applications. Its other focus is locality of effects: the property that in many of the same applications, typical imperative commands only affect small, local regions in the shared heap.


title: Software evolution and the moving picture metaphor

abstract: Software systems evolve over time. Currently we do not do a good job of documenting this evolution. This essay discusses the need to better document software evolution and introduces the Moving Picture Metaphor. Source Control Management systems are more like collections of still photographs than moving pictures. Still photography is not ideal when trying to capture evolutional changes. Moving pictures do a much better job. A storyteller can use moving pictures to tell compelling stories that are easier to digest than traditional documentation. We can learn a great deal from watching stories that document a system's evolution.


title: CZ: multiple inheritance without diamonds

authors:
Donna Malayeri
Jonathan Aldrich

abstract: Multiple inheritance has long been plagued with the "diamond" inheritance problem, leading to solutions that restrict expressiveness, such as mixins and traits. Instead, we address the diamond problem directly, considering two difficulties it causes: ensuring a correct semantics for object initializers, and typechecking multiple dispatch in a modular fashion-the latter problem arising even with multiple interface inheritance. We show that previous solutions to these problems are either unsatisfactory or cumbersome, and suggest a novel approach: supporting multiple inheritance but forbidding diamond inheritance. Expressiveness is retained through two features: a "requires" construct that provides a form of subtyping without inheritance (inspired by Scala), and a dynamically-dispatched "super" call similar to that found in traits. Through examples, we illustrate that inheritance diamonds can be eliminated via a combination of "requires" and ordinary inheritance. We provide a sound formal model for our language and demonstrate its modularity and expressiveness.


title: Flapjax: a programming language for Ajax applications

authors:
Leo A. Meyerovich
Arjun Guha
Jacob P. Baskin
Gregory H. Cooper
Michael Greenberg
Aleks Bromfield
Shriram Krishnamurthi

abstract: This paper presents Flapjax, a language designed for contemporary Web applications. These applications communicate with servers and have rich, interactive interfaces. Flapjax provides two key features that simplify writing these applications. First, it provides event streams, a uniform abstraction for communication within a program as well as with external Web services. Second, the language itself is reactive: it automatically tracks data dependencies and propagates updates along those dataflows. This allows developers to write reactive interfaces in a declarative and compositional style.

awards: Most Influential OOPSLA Paper (awarded 2019)


title: Writing code for other people: cognitive psychology and the fundamentals of good software design principles

abstract: This paper demonstrates how the cognitive model of the mind can explain the core fundamentals behind widely accepted design principles. The conclusion is that software design is largely a task of chunking analogies and presents a theory that is detailed enough to be accessible to even the most inexperienced programmer. The corollary of which is a pedagogical approach to understanding design principles rather than the necessity of years of software development experience.


title: Inferred call path profiling

authors:
Todd Mytkowicz
Devin Coughlin
Amer Diwan

abstract: Prior work has found call path profiles to be useful for optimizers and programmer-productivity tools. Unfortunately, previous approaches for collecting path profiles are expensive: they need to either execute additional instructions (to track calls and returns) or they need to walk the stack. The state-of-the-art techniques for call path profiling slow down the program by 7% (for C programs) and 20% (for Java programs). This paper describes an innovative technique that collects minimal information from the running program and later (offline) infers the full call paths from this information.


title: Demystifying model transformations: an approach based on automated rule inference

authors:
Mangala Gowri Nanda
Senthil Mani
Vibha Singhal Sinha
Saurabh Sinha

abstract: Model-driven development (MDD) is widely used to develop modern business applications. MDD involves creating models at different levels of abstractions. Starting with models of domain concepts, these abstractions are successively refined, using transforms, to design-level models and, eventually, code-level artifacts. Although many tools exist that support transform creation and verification, tools that help users in understanding and using transforms are rare. In this paper, we present an approach for assisting users in understanding model transformations and debugging their input models. We use automated program-analysis techniques to analyze the transform code and compute constraints under which a transformation may fail or be incomplete. These code-level constraints are mapped to the input model elements to generate model-level rules. The rules can be used to validate whether an input model violates transform constraints, and to support general user queries about a transformation. We have implemented the analysis in a tool called XYLEM. We present empirical results, which indicate that (1) our approach can be effective in inferring useful rules, and (2) the rules let users efficiently diagnose a failing transformation without examining the transform source code.


title: NUMA-aware memory manager with dominant-thread-based copying GC

abstract: We propose a novel online method of identifying the preferred NUMA nodes for objects with negligible overhead during the garbage collection time as well as object allocation time. Since the number of CPUs (or NUMA nodes) is increasing recently, it is critical for the memory manager of the runtime environment of an object-oriented language to exploit the low latency of local memory for high performance. To locate the CPU of a thread that frequently accesses an object, prior research uses the runtime information about memory accesses as sampled by the hardware. However, the overhead of this approach is high for a garbage collector.


title: Regrowing a language: refactoring tools allow programming languages to evolve

authors:
Jeffrey L. Overbey
Ralph E. Johnson

abstract: Successful programming languages change as they age. They tend to become more complex, and eventually some features become outdated or are rarely used. Programming tools for these languages become more complex as well, since they have to support archaic features. Old programs are hard to maintain, since these archaic features are unfamiliar to modern programmers. These problems can be solved by refactoring tools that can transform programs to use the modern form. We show that refactoring tools can ease the cost of program evolution by examining the evolution of two languages, Fortran and Java, and showing that each change corresponds to an automatable refactoring.


title: Executing code in the past: efficient in-memory object graph versioning

authors:
Frédéric Pluquet
Stefan Langerman
Roel Wuyts

abstract: Object versioning refers to how an application can have access to previous states of its objects. Implementing this mechanism is hard because it needs to be efficient in space and time, and well integrated with the programming language. This paper presents HistOOry, an object versioning system that uses an efficient data structure to store and retrieve past states. It needs only three primitives, and existing code does not need to be modified to be versioned. It provides fine-grained control over what parts of objects are versioned and when. It stores all states, past and present, in memory. Code can be executed in the past of the system and will see the complete system at that point in time. We have implemented our model in Smalltalk and used it for three applications that need versioning: checked postconditions, stateful execution tracing and a planar point location implementation. Benchmarks are provided to asses the practical complexity of our implementation.


title: Agile anthropology and Alexander's architecture: an essay in three voices

authors:
Jenny Quillien
Pam Rostal
Dave West

abstract: During its formative decades the software community looked twice to the theories of Christopher Alexander for inspiration, both times failing to completely master the architect's most useful insights. Now a third opportunity presents itself with Alexander's recent publication, The Nature of Order. Serious apprenticeship, however, imposes a prerequisite of sober self-reflection and evaluation. What, really, is the nature of the developer's tasks? Under what philosophical umbrella has the software community matured until now? Do other philosophical traditions offer alternative and perhaps more pertinent epistemologies? What voices, besides Alexander's, might contribute to the community's evolution? We address these questions along with theory building, ethnography, weak links, design heuristics, agility, and complex systems, all of which combine with Alexander's new theories to suggest different ways of doing what we do, better.


title: Design pattern density defined

abstract: Design pattern density is a metric that measures how much of an object-oriented design can be understood and represented as instances of design patterns. Expert developers have long believed that a high design pattern density implies a high maturity of the design under inspection. This paper presents a quantifiable and observable definition of this metric. The metric is illustrated and qualitatively validated using four real-world case studies. We present several hypotheses of the metric's meaning and their implications, including the one about design maturity. We propose that the design pattern density of a maturing framework has a fixed point and we show that if software design patterns make learning frameworks easier, a framework's design pattern density is a measure of how much easier it will become.


title: Self type constructors

authors:
Chieri Saito
Atsushi Igarashi

abstract: Bruce and Foster proposed the language LOOJ, an extension of Java with the notion of MyType, which represents the type of a self reference and changes its meaning along with inheritance. MyType is useful to write extensible yet type-safe classes for objects with recursive interfaces, that is, ones with methods that take or return objects of the same type as the receiver.


title: Enhancing source-level programming tools with an awareness of transparent program transformations

authors:
Myoungkyu Song
Eli Tilevich

abstract: Programs written in managed languages are compiled to a platform-independent intermediate representation, such as Java bytecode. The relative high level of Java bytecode has engendered a widespread practice of changing the bytecode directly, without modifying the maintained version of the source code. This practice, called bytecode engineering or enhancement, has become indispensable in introducing various concerns, including persistence, distribution, and security, transparently. For example, transparent persistence architectures help avoid the entanglement of business and persistence logic in the source code by changing the bytecode directly to synchronize objects with stable storage. With functionality added directly at the bytecode level, the source code reflects only partial semantics of the program. Specifically, the programmer can neither ascertain the program's runtime behavior by browsing its source code, nor map the runtime behavior back to the original source code.


title: Minimizing dependencies within generic classes for faster and smaller programs

authors:
Dan Tsafrir
Robert W. Wisniewski
David F. Bacon
Bjarne Stroustrup

abstract: Generic classes can be used to improve performance by allowing compile-time polymorphism. But the applicability of compile-time polymorphism is narrower than that of runtime polymorphism, and it might bloat the object code. We advocate a programming principle whereby a generic class should be implemented in a way that minimizes the dependencies between its members (nested types, methods) and its generic type parameters. Conforming to this principle (1) reduces the bloat and (2) gives rise to a previously unconceived manner of using the language that expands the applicability of compile-time polymorphism to a wider range of problems. Our contribution is thus a programming technique that generates faster and smaller programs. We apply our ideas to GCC's STL containers and iterators, and we demonstrate notable speedups and reduction in object code size (real application runs 1.2x to 2.1x faster and STL code is 1x to 25x smaller). We conclude that standard generic APIs (like STL) should be amended to reflect the proposed principle in the interest of efficiency and compactness. Such modifications will not break old code, simply increase flexibility. Our findings apply to languages like C++, C#, and D, which realize generic programming through multiple instantiations.


title: Allocation wall: a limiting factor of Java applications on emerging multi-core platforms

authors:
Yi Zhao
Jin Shi
Kai Zheng
Haichuan Wang
Haibo Lin
Ling Shao

abstract: Multi-core processors are widely used in computer systems. As the performance of microprocessors greatly exceeds that of memory, the memory wall becomes a limiting factor. It is important to understand how the large disparity of speed between processor and memory influences the performance and scalability of Java applications on emerging multi-core platforms.


title: The Tower of Babel did not fail

authors:
Paul Adamczyk
Munawar Hafiz

abstract: Fred Brooks' retelling of the biblical story of the Tower of Babel offers many insights into what makes building software difficult. The difficulty, according to common interpretations, comes from the communication and organizational problems in software development. But the story contains one more important lesson that people tend to miss: one cannot accomplish impossible goals, which programmers are often asked to do. Software engineering, as a discipline, can overcome poor communication; but as long as we attempt to live up to impossible expectations, we will always fail.


title: Performance analysis of idle programs

authors:
Erik R. Altman
Matthew Arnold
Stephen Fink
Nick Mitchell

abstract: This paper presents an approach for performance analysis of modern enterprise-class server applications. In our experience, performance bottlenecks in these applications differ qualitatively from bottlenecks in smaller, stand-alone systems. Small applications and benchmarks often suffer from CPU-intensive hot spots. In contrast, enterprise-class multi-tier applications often suffer from problems that manifest not as hot spots, but as idle time, indicating a lack of forward motion. Many factors can contribute to undesirable idle time, including locking problems, excessive system-level activities like garbage collection, various resource constraints, and problems driving load.We present the design and methodology for WAIT, a tool to diagnosis the root cause of idle time in server applications. Given lightweight samples of Java activity on a single tier, the tool can often pinpoint the primary bottleneck on a multi-tier system. The methodology centers on an informative abstraction of the states of idleness observed in a running program. This abstraction allows the tool to distinguish, for example, between hold-ups on a database machine, insufficient load, lock contention in application code, and a conventional bottleneck due to a hot method. To compute the abstraction, we present a simple expert system based on an extensible set of declarative rules.WAIT can be deployed on the fly, without modifying or even restarting the application. Many groups in IBM have applied the tool to diagnosis performance problems in commercial systems, and we present a number of examples as case studies.


title: Managing ambiguity in programming by finding unambiguous examples

authors:
Kenneth C. Arnold
Henry Lieberman

abstract: We propose a new way to raise the level of discourse in the programming process: permit ambiguity, but manage it by linking it to unambiguous examples. This allows programming environments to work with high-level descriptions that lack precise semantics, such as natural language descriptions or conceptual diagrams, without requiring programmers to formulate their ideas in a formal language first. As an example of this idea, we present Zones, a code search and reuse interface that connects code with ambiguous natural language annotations about its purpose. The backend, called ProcedureSpace, induces relationships between these purpose annotations, static code analysis features, and a variety of natural language background knowledge. ProcedureSpace can search for code given purpose descriptions or vice versa, and can even find code that was never annotated or commented. Since completed Zones searches become annotations, the system learns from user interaction. Users in a preliminary study found that reasoning jointly over natural language and programming language helped them reuse code.


title: Lime: a Java-compatible and synthesizable language for heterogeneous architectures

authors:
Joshua S. Auerbach
David F. Bacon
Perry Cheng
Rodric M. Rabbah

abstract: The halt in clock frequency scaling has forced architects and language designers to look elsewhere for continued improvements in performance. We believe that extracting maximum performance will require compilation to highly heterogeneous architectures that include reconfigurable hardware.


title: SPUR: a trace-based JIT compiler for CIL

authors:
Michael Bebenita
Florian Brandner
Manuel Fähndrich
Francesco Logozzo
Wolfram Schulte
Nikolai Tillmann
Herman Venter

abstract: Tracing just-in-time compilers (TJITs) determine frequently executed traces (hot paths and loops) in running programs and focus their optimization effort by emitting optimized machine code specialized to these traces. Prior work has established this strategy to be especially beneficial for dynamic languages such as JavaScript, where the TJIT interfaces with the interpreter and produces machine code from the JavaScript trace.


title: Concurrent programming with revisions and isolation types

authors:
Sebastian Burckhardt
Alexandro Baldassin
Daan Leijen

abstract: Building applications that are responsive and can exploit parallel hardware while remaining simple to write, understand, test, and maintain, poses an important challenge for developers. In particular, it is often desirable to enable various tasks to read or modify shared data concurrently without requiring complicated locking schemes that may throttle concurrency and introduce bugs.


title: Tribal ownership

authors:
Nicholas Cameron
James Noble
Tobias Wrigstad

abstract: Tribal Ownership unifies class nesting and object ownership. Tribal Ownership is based on Tribe, a language with nested classes and object families. In Tribal Ownership, a program's runtime object ownership structure is characterised by the lexical nesting structure of its classes.We build on a variant of Tribe to present a descriptive ownership system, using object nesting to describe heap partitions, but without imposing any restrictions on programming disciplines. We then demonstrate how a range of different prescriptive ownership policies can be supported on top of the descriptive Tribal Ownership mechanism; including a novel owners-as-local-dominators policy. We formalise our type system and prove soundness and several ownership invariants. The resulting system requires strikingly few annotations, and uses well-understood encapsulation techniques to create ownership systems that should be intuitive for programmers.


title: Language virtualization for heterogeneous parallel computing

authors:
Hassan Chafi
Zach DeVito
Adriaan Moors
Tiark Rompf
Arvind K. Sujeeth
Pat Hanrahan
Martin Odersky
Kunle Olukotun

abstract: As heterogeneous parallel systems become dominant, application developers are being forced to turn to an incompatiblemix of low level programming models (e.g. OpenMP, MPI, CUDA, OpenCL). However, these models do little to shield developers from the difficult problems of parallelization, data decomposition and machine-specific details. Most programmersare having a difficult time using these programming models effectively. To provide a programming modelthat addresses the productivity and performance requirements for the average programmer, we explore a domainspecificapproach to heterogeneous parallel programming.


title: Mostly modular compilation of crosscutting concerns by contextual predicate dispatch

authors:
Shigeru Chiba
Atsushi Igarashi
Salikh Zakirov

abstract: The modularity of aspect-oriented programming (AOP) has been a controversial issue. To investigate this issue compared with object-oriented programming (OOP), we propose a simple language providing AOP mechanisms, which are enhanced traditional OOP mechanisms. We also present its formal system and then show that programs in this language can be only mostly modularly (i.e. separately) typechecked and compiled.We mention a source of this unmodularity and discuss whether or not it is appropriate to claim that AOP breaks modularity compared with OOP.


title: Registration-based language abstractions

authors:
Samuel Davis
Gregor Kiczales

abstract: Programming language innovation has been hindered by the difficulty of making changes to existing languages. A key source of difficulty is the tyrannical nature of existing approaches to realizing languages -- adding a new language construct means that any tool, document or programmer that works with the language must be prepared to deal with that construct.A registration-based approach makes it possible to define language constructs that are not tyrannical. They are instead transient -- the program appears to be written using the constructs only so long as a given programmer wants to see it that way. This approach may have the potential to greatly facilitate programming language innovation.


title: Symbolic heap abstraction with demand-driven axiomatization of memory invariants

authors:
Isil Dillig
Thomas Dillig
Alex Aiken

abstract: Many relational static analysis techniques for precise reasoning about heap contents perform an explicit case analysis of all possible heaps that can arise. We argue that such precise relational reasoning can be obtained in a more scalable and economical way by enforcing the memory invariant that every concrete memory location stores one unique value directly on the heap abstraction. Our technique combines the strengths of analyses for precise reasoning about heap contents with approaches that prioritize axiomatization of memory invariants, such as the theory of arrays. Furthermore, by avoiding an explicit case analysis, our technique is scalable and powerful enough to analyze real-world programs with intricate use of arrays and pointers; in particular, we verify the absence of buffer overruns, incorrect casts, and null pointer dereferences in OpenSSH (over 26,000 lines of code) after fixing 4 previously undiscovered bugs found by our system. Our experiments also show that the combination of reasoning about heap contents and enforcing existence and uniqueness invariants is crucial for this level of precision. Copyright © 2010 ACM.


title: To upgrade or not to upgrade: impact of online upgrades across multiple administrative domains

authors:
Tudor Dumitras
Priya Narasimhan
Eli Tilevich

abstract: Online software upgrades are often plagued by runtime behaviors that are poorly understood and difficult to ascertain. For example, the interactions among multiple versions of the software expose the system to race conditions that can introduce latent errors or data corruption. Moreover, industry trends suggest that online upgrades are currently needed in large-scale enterprise systems, which often span multiple administrative domains (e.g., Web 2.0 applications that rely on AJAX client-side code or systems that lease cloud-computing resources). In such systems, the enterprise does not control all the tiers of the system and cannot coordinate the upgrade process, making existing techniques inadequate to prevent mixed-version races. In this paper, we present an analytical framework for impact assessment, which allows system administrators to directly compare the risk of following an online-upgrade plan with the risk of delaying or canceling the upgrade. We also describe an executable model that implements our formal impact assessment and enables a systematic approach for deciding whether an online upgrade is appropriate. Our model provides a method of last resort for avoiding undesirable program behaviors, in situations where mixed-version races cannot be avoided through other technical means.


title: The case for evolvable software

abstract: As programmers, we like to think of software as the product of our intelligent design, carefully crafted to meet well-specified goals. In reality, software evolves inadvertently through the actions of many individual programmers, often leading to unanticipated consequences. Large complex software systems are subject to constraints similar to those faced by evolving biological systems, and we have much to gain by viewing software through the lens of evolutionary biology. The talk will highlight recent research that applies the mechanisms of evolution quite directly to the problem of repairing software bugs.


title: Scalable and systematic detection of buggy inconsistencies in source code

authors:
Mark Gabel
Junfeng Yang
Yuan Yu
Moisés Goldszmidt
Zhendong Su

abstract: Software developers often duplicate source code to replicate functionality. This practice can hinder the maintenance of a software project: bugs may arise when two identical code segments are edited inconsistently. This paper presents DejaVu, a highly scalable system for detecting these general syntactic inconsistency bugs.


title: Better science through art

authors:
Richard P. Gabriel
Kevin J. Sullivan

abstract: How do artists and scientists work? The same.


title: An experiment about static and dynamic type systems: doubts about the positive impact of static type systems on development time

abstract: Although static type systems are an essential part in teach-ing and research in software engineering and computer science, there is hardly any knowledge about what the impact of static type systems on the development time or the resulting quality for a piece of software is. On the one hand there are authors that state that static type systems decrease an application's complexity and hence its development time (which means that the quality must be improved since developers have more time left in their projects). On the other hand there are authors that argue that static type systems increase development time (and hence decrease the code quality) since they restrict developers to express themselves in a desired way. This paper presents an empirical study with 49 subjects that studies the impact of a static type system for the development of a parser over 27 hours working time. In the experiments the existence of the static type system has neither a positive nor a negative impact on an application's development time (under the conditions of the experiment).


title: Faith, hope, and love: an essay on software science's neglect of human factors

abstract: Research in the area of programming languages has different facets -- from formal reasoning about new programming language constructs (such as type soundness proofs for new type systems) over inventions of new abstractions, up to performance measurements of virtual machines. A closer look into the underlying research methods reveals a distressing characteristic of programming language research: developers, which are the main audience for new language constructs, are hardly considered in the research process. As a consequence, it is simply not possible to state whether a new construct that requires some kind of interaction with the developer has any positive impact on the construction of software. This paper argues for appropriate research methods in programming language research that rely on studies of developers -- and argues that the introduction of corresponding empirical methods not only requires a new understanding of research but also a different view on how to teach software science to students.


title: Dynamic parallelization of recursive code: part 1: managing control flow interactions with the continuator

authors:
Charlotte Herzeel
Pascal Costanza

abstract: While most approaches to automatic parallelization focus on compilation approaches for parallelizing loop iterations, we advocate the need for new virtual machines that can parallelize the execution of recursive programs. In this paper, we show that recursive programs can be effectively parallelized when arguments to procedures are evaluated concurrently and branches of conditional statements are speculatively executed in parallel. We introduce the continuator concept, a runtime structure that tracks and manages the control dependences between such concurrently spawned tasks, ensuring adherence to the sequential semantics of the parallelized program. As a proof of concept, we discuss the details of a parallel interpreter for Scheme (implemented in Common Lisp) based on these ideas, and show the results from executing the Clinger benchmark suite for Scheme.


title: Agility in context

authors:
Rashina Hoda
Philippe Kruchten
James Noble
Stuart Marshall

abstract: Evangelists for Agile methods strongly encourage all projects to follow every practice of their chosen method. Based on a Grounded Theory study involving 40 participants at 16 organizations, and corroborated by 4 independent case studies, we argue that development methods and practices must be adapted to fit their contexts. Understanding Agility in context will help development teams, their managers, and Agile coaches to adapt development processes to fit their projects' contexts.


title: A simple inductive synthesis methodology and its applications

authors:
Shachar Itzhaky
Sumit Gulwani
Neil Immerman
Mooly Sagiv

abstract: Given a high-level specification and a low-level programming language, our goal is to automatically synthesize an efficient program that meets the specification. In this paper, we present a new algorithmic methodology for inductive synthesis that allows us to do this.


title: Instrumentation and sampling strategies for cooperative concurrency bug isolation

authors:
Guoliang Jin
Aditya V. Thakur
Ben Liblit
Shan Lu

abstract: Fixing concurrency bugs (or crugs) is critical in modern software systems. Static analyses to find crugs such as data races and atomicity violations scale poorly, while dynamic approaches incur high run-time overheads. Crugs manifest only under specific execution interleavings that may not arise during in-house testing, thereby demanding a lightweight program monitoring technique that can be used post-deployment. We present Cooperative Crug Isolation (CCI), a lowoverhead instrumentation framework to diagnose productionrun failures caused by crugs. CCI tracks specific thread interleavings at run-time, and uses statistical models to identify strong failure predictors among these. We offer a varied suite of predicates that represent different trade-offs between complexity and fault isolation capability. We also develop variant random sampling strategies that suit different types of predicates and help keep the run-time overhead low. Experiments with 9 real-world bugs in 6 non-trivial C applications show that these schemes span a wide spectrum of performance and diagnosis capabilities, each suitable for different usage scenarios.


title: Refactoring references for library migration

authors:
Puneet Kapur
Bradley Cossette
Robert J. Walker

abstract: Automated refactoring is a key feature of modern IDEs. Existing refactorings rely on the transformation of source code declarations, in which references may also be transformed as a side effect. However, there exist situations in which a declaration is not available for refactoring or would be inappropriate to transform, for example, in the presence of dangling references or where a set of references should be retargeted to a different declaration.


title: The spoofax language workbench: rules for declarative specification of languages and IDEs

authors:
Lennart C. L. Kats
Eelco Visser

abstract: Spoofax is a language workbench for efficient, agile development of textual domain-specific languages with state-of-the-art IDE support. Spoofax integrates language processing techniques for parser generation, meta-programming, and IDE development into a single environment. It uses concise, declarative specifications for languages and IDE services. In this paper we describe the architecture of Spoofax and introduce idioms for high-level specifications of language semantics using rewrite rules, showing how analyses can be reused for transformations, code generation, and editor services such as error marking, reference resolving, and content completion. The implementation of these services is supported by language-parametric editor service classes that can be dynamically loaded by the Eclipse IDE, allowing new languages to be developed and used side-by-side in the same Eclipse environment.

awards: Most Influential OOPSLA Paper (awarded 2020)


title: Pure and declarative syntax definition: paradise lost and regained

authors:
Lennart C. L. Kats
Eelco Visser
Guido Wachsmuth

abstract: Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained.


title: Component adaptation and assembly using interface relations

abstract: Software's expense owes partly to frequent reimplementation of similar functionality and partly to maintenance of patches, ports or components targeting evolving interfaces. More modular non-invasive approaches are unpopular because they entail laborious wrapper code. We propose Cake, a rule-based language describing compositions using interface relations. To evaluate it, we compare several existing wrappers with reimplemented Cake versions, finding the latter to be simpler and better modularised.


title: Random testing for higher-order, stateful programs

authors:
Casey Klein
Matthew Flatt
Robert Bruce Findler

abstract: Testing is among the most effective tools available for finding bugs. Still, we know of no automatic technique for generating test cases that expose bugs involving a combination of mutable state and callbacks, even though objects and method overriding set up exactly that combination. For such cases, a test generator must create callbacks or subclasses that aggressively exercise side-effecting operations using combinations of generated objects.This paper presents a new algorithm for randomly testing programs that use state and callbacks. Our algorithm exploits a combination of contracts and environment bindings to guide the test-case generator toward interesting inputs. Our prototype implementation for Racket (formerly PLT Scheme) - which has a Java-like class system, but with first-class classes as well as gbeta-like augmentable methods - uncovered dozens of bugs in a well-tested and widely used text-editor library.We describe our approach in a precise, formal notation, borrowing the techniques used to describe operational semantics and type systems. The formalism enables us to provide a compact and self-contained explanation of the core of our technique without the ambiguity usually present in pseudo-code descriptions.


title: Modular logic metaprogramming

authors:
Karl Klose
Klaus Ostermann

abstract: In logic metaprogramming, programs are not stored as plain textfiles but rather derived from a deductive database. While the benefits of this approach for metaprogramming are obvious, its incompatibility with separate checking limits its applicability to large-scale projects. We analyze the problems inhibiting separate checking and propose a class of logics that reconcile logic metaprogramming and separate checking. We have formalized the resulting module system and have proven the soundness of separate checking. We validate its feasibility by presenting the design and implementation of a specific logic that is able to express many metaprogramming examples from the literature.


title: From OO to FPGA: fitting round objects into square hardware?

authors:
Stephen Kou
Jens Palsberg

abstract: Consumer electronics today such as cell phones often have one or more low-power FPGAs to assist with energyintensive operations in order to reduce overall energy consumption and increase battery life. However, current techniques for programming FPGAs require people to be specially trained to do so. Ideally, software engineers can more readily take advantage of the benefits FPGAs offer by being able to program them using their existing skills, a common one being object-oriented programming. However, traditional techniques for compiling object-oriented languages are at odds with today's FPGA tools, which support neither pointers nor complex data structures. Open until now is the problem of compiling an object-oriented language to an FPGA in a way that harnesses this potential for huge energy savings. In this paper, we present a new compilation technique that feeds into an existing FPGA tool chain and produces FPGAs with up to almost an order of magnitude in energy savings compared to a low-power microprocessor while still retaining comparable performance and area usage.


title: Task types for pervasive atomicity

authors:
Aditya Kulkarni
Yu David Liu
Scott F. Smith

abstract: Atomic regions are an important concept in correct concurrent programming: since atomic regions can be viewed as having executed in a single step, atomicity greatly reduces the number of possible interleavings the programmer needs to consider. This paper describes a method for building atomicity into a programming language in an organic fashion. We take the view that atomicity holds for whole threads by default, and a division into smaller atomic regions occurs only at points where an explicit need for sharing is needed and declared. A corollary of this view is every line of code is part of some atomic region. We define a polymorphic type system, Task Types, to enforce most of the desired atomicity properties statically. We show the reasonableness of our type system by proving that type soundness, isolation invariance, and atomicity enforcement properties hold at run time. We also present initial results of a Task Types implementation built on Java


title: Supporting dynamic, third-party code customizations in JavaScript using aspects

authors:
Benjamin S. Lerner
Herman Venter
Dan Grossman

abstract: Web sites and web browsers have recently evolved into platforms on top of which entire applications are delivered dynamically, mostly as JavaScript source code. This delivery format has sparked extremely enthusiastic efforts to customize both individual web sites and entire browsers in ways the original authors never expected or accommodated. Such customizations take the form of yet more script dynamically injected into the application, and the current idioms to do so exploit arcane JavaScript features and are extremely brittle. In this work, we accept the popularity of extensions and seek better linguistic mechanisms to support them.


title: G-Finder: routing programming questions closer to the experts

authors:
Wei Li
Charles Zhang
Songlin Hu

abstract: Programming forums are becoming the primary tools for programmers to find answers for their programming problems. Our empirical study of popular programming forums shows that the forum users experience long waiting period for answers and a small number of experts are often overloaded with questions. To improve the usage experience, we have designed and implemented G-Finder, both an algorithm and a tool that makes intelligent routing decisions as to which participant is the expert for answering a particular programming question. Our main approach is to leverage the source code information of the software systems that forums are dedicated to, and discover latent relationships between forums users. Our algorithms construct the concept networks and the user networks from the program source and the forum data. We use programming questions to dynamically integrate these two networks and present an adaptive ranking of the potential experts. Our evaluation of G-Finder, using the data from three large programming forums, takes a retrospective view to check if G-Finder can correctly predict the experts who provided answers to programming questions. The evaluation results show that G-Finder improves the prediction precision by 25% to 74%, compared to related approaches.


title: A dynamic evaluation of the precision of static heap abstractions

authors:
Percy Liang
Omer Tripp
Mayur Naik
Mooly Sagiv

abstract: The quality of a static analysis of heap-manipulating programs is largely determined by its heap abstraction. Object allocation sites are a commonly-used abstraction, but are too coarse for some clients. The goal of this paper is to investigate how various refinements of allocation sites can improve precision. In particular, we consider abstractions that use call stack, object recency, and heap connectivity information. We measure the precision of these abstractions dynamically for four different clients motivated by concurrency and on nine Java programs chosen from the DaCapo benchmark suite. Our dynamic results shed new light on aspects of heap abstractions that matter for precision, which allows us to more effectively navigate the large space of possible heap abstractions


title: A time-aware type system for data-race protection and guaranteed initialization

authors:
Nicholas D. Matsakis
Thomas R. Gross

abstract: We introduce a type system based on intervals, objects representing the time in which a block of code will execute. The type system can verify time-based properties such as when a field will be accessed or a method will be invoked. One concrete application of our type system is data-race protection: For fields which are initialized during one phase of the program and constant thereafter, users can designate the interval during which the field is mutable. Code which happens after this initialization interval can safely read the field in parallel. We also support fields guarded by a lock and even the use of dynamic race detectors. Another use for intervals is to designate different phases in the object’s lifetime, such as a constructor phase. The type system then ensures that only appropriate methods are invoked in each phase.


title: The two-state solution: native and serializable continuations accord

abstract: Continuation-based Web servers provide advantages over traditional Web application development through the increase of expressive power they allow. This leads to fewer errors and more productivity for the programmers that adopt them. Unfortunately, existing implementation techniques force a hard choice between scalability and expressiveness.


title: Hera-JVM: a runtime system for heterogeneous multi-core architectures

authors:
Ross McIlroy
Joe Sventek

abstract: Heterogeneous multi-core processors, such as the IBM Cell processor, can deliver high performance. However, these processors are notoriously difficult to program: different cores support different instruction set architectures, and the processor as a whole does not provide coherence between the different cores’ local memories. We present Hera-JVM, an implementation of the Java Virtual Machine which operates over the Cell processor, thereby making this platforms more readily accessible to mainstream developers. Hera-JVM supports the full Java language; threads from an unmodified Java application can be simultaneously executed on both the main PowerPC-based core and on the additional SPE accelerator cores. Migration of threads between these cores is transparent from the point of view of the application, requiring no modification to Java source code or bytecode. Hera-JVM supports the existing Java Memory Model, even though the underlying hardware does not provide cache coherence between the different core types. We examine Hera-JVM’s performance under a series of real-world Java benchmarks from the SpecJVM, Java Grande and Dacapo benchmark suites. These benchmarks show a wide variation in relative performance on the different core types of the Cell processor, depending upon the nature of their workload. Execution of these benchmarks on Hera-JVM can achieve speedups of up to 2.25x by using one of the Cell processor’s SPE accelerator cores, compared to execution on the main PowerPC-based core. When all six SPE cores are exploited, parallel workloads can achieve speedups of up to 13x compared to execution on the single PowerPC core.


title: Parallel inclusion-based points-to analysis

authors:
Mario Méndez-Lojo
Augustine Mathew
Keshav Pingali

abstract: Inclusion-based points-to analysis provides a good trade-off between precision of results and speed of analysis, and it has been incorporated into several production compilers including gcc. There is an extensive literature on how to speed up this algorithm using heuristics such as detecting and collapsing cycles of pointer-equivalent variables. This paper describes a complementary approach based on exploiting parallelism. Our implementation exploits two key insights. First, we show that inclusion-based points-to analysis can be formulated entirely in terms of graphs and graph rewrite rules. This exposes the amorphous data-parallelism in this algorithm and makes it easier to develop a parallel implementation. Second, we show that this graph-theoretic formulation reveals certain key properties of the algorithm that can be exploited to obtain an efficient parallel implementation. Our parallel implementation achieves a scaling of up to 3x on a 8-core machine for a suite of ten large C programs. For all but the smallest benchmarks, the parallel analysis outperforms a state-of-the-art, highly optimized, serial implementation of the same algorithm. To the best of our knowledge, this is the first parallel implementation of a points-to analysis.


title: A domain-specific approach to architecturing error handling in pervasive computing

authors:
Julien Mercadal
Quentin Enard
Charles Consel
Nicolas Loriant

abstract: The challenging nature of error handling constantly escalates as a growing number of environments consists of networked devices and software components. In these environments, errors cover a uniquely large spectrum of situations related to each layer ranging from hardware to distributed platforms, to software components. Handling errors becomes a daunting task for programmers, whose outcome is unpredictable. Scaling up error handling requires to raise the level of abstraction beyond the code level and the try-catch construct, approaching error handling at the software architecture level.


title: A graph-based approach to API usage adaptation

authors:
Hoan Anh Nguyen
Tung Thanh Nguyen
Gary Wilson Jr.
Anh Tuan Nguyen
Miryung Kim
Tien N. Nguyen

abstract: Reusing existing library components is essential for reducing the cost of software development and maintenance. When library components evolve to accommodate new feature requests, to fix bugs, or to meet new standards, the clients of software libraries often need to make corresponding changes to correctly use the updated libraries. Existing API usage adaptation techniques support simple adaptation such as replacing the target of calls to a deprecated API, however, cannot handle complex adaptations such as creating a new object to be passed to a different API method, or adding an exception handling logic that surrounds the updated API method calls. This paper presents LIBSYNC that guides developers in adapting API usage code by learning complex API usage adaptation patterns from other clients that already migrated to a new library version (and also from the API usages within the library’s test code). LIBSYNC uses several graph-based techniques (1) to identify changes to API declarations by comparing two library versions, (2) to extract associated API usage skeletons before and after library migration, and (3) to compare the extracted API usage skeletons to recover API usage adaptation patterns. Using the learned adaptation patterns, LIBSYNC recommends the locations and edit operations for adapting API usages. The evaluation of LIBSYNC on real-world software systems shows that it is highly correct and useful with a precision of 100 % and a recall of 91%.


title: A study of Java's non-Java memory

authors:
Kazunori Ogata
Dai Mikurube
Kiyokuni Kawachiya
Scott Trent
Tamiya Onodera

abstract: A Java application sometimes raises an out-of-memory ex-ception. This is usually because it has exhausted the Java heap. However, a Java application can raise an out-of-memory exception when it exhausts the memory used by Java that is not in the Java heap. We call this area non-Java memory. For example, an out-of-memory exception in the non-Java memory can happen when the JVM attempts to load too many classes. Although it is relatively rare to ex-haust the non-Java memory compared to exhausting the Java heap, a Java application can consume a considerable amount of non-Java memory.This paper presents a quantitative analysis of non-Java memory. To the best of our knowledge, this is the first in-depth analysis of the non-Java memory. To do this we cre-ated a tool called Memory Analyzer for Redundant, Unused, and String Areas (MARUSA), which gathers memory statis-tics from both the OS and the Java virtual machine, break-ing down and visualizing the non-Java memory usage.We studied the use of non-Java memory for a wide range of Java applications, including the DaCapo benchmarks and Apache DayTrader. Our study is based on the IBM J9 Java Virtual Machine for Linux. Although some of our results may be specific to this combination, we believe that most of our observations are applicable to other platforms as well.


title: Type classes as objects and implicits

authors:
Bruno C. d. S. Oliveira
Adriaan Moors
Martin Odersky

abstract: Type classes were originally developed in Haskell as a disciplined alternative to ad-hoc polymorphism. Type classes have been shown to provide a type-safe solution to important challenges in software engineering and programming languages such as, for example, retroactive extension of programs. They are also recognized as a good mechanism for concept-based generic programming and, more recently, have evolved into a mechanism for type-level computation.


title: Flexible modeling tools for pre-requirements analysis: conceptual architecture and research challenges

authors:
Harold Ossher
Rachel K. E. Bellamy
Ian Simmonds
David Amid
Ateret Anaby-Tavor
Matthew Callery
Michael Desmond
Jacqueline de Vries
Amit Fisher
Sophia Krasikov

abstract: A serious tool gap exists at the start of the software lifecy-cle, before requirements formulation. Pre-requirements analysts gather information, organize it to gain insight, en-vision possible futures, and present insights and recom-mendations to stakeholders. They typically use office tools, which give great freedom, but no help with consistency management, change propagation, or information migration to downstream tools. Despite these downsides, office tools are still favored over modeling tools, which are constrain-ing and difficult to use. We introduce the notion of flexible modeling tools, which blend the advantages of office and modeling tools. We propose a conceptual architecture for such tools, and outline research challenges to be met in realizing them. We briefly describe the Business Insight Toolkit, a prototype tool embodying this architecture.


title: Art, science, and fear

abstract: Scientists and artists share a conviction that key elements of their work-creativity, craft, analysis, intuition--are somehow deeply similar. Most Onward! participants will have a clear sense of the scientific variants of these qualities, and of what it takes to manifest them in our day-to-day work as researchers and software designers. This talk looks at how ideas are developed and refined in the making of visual art, how the same qualities--especially creativity, the most ineffable--come into play, and how artists and scientists grapple with the various fears that beset creative work.


title: Monitor optimization via stutter-equivalent loop transformation

authors:
Rahul Purandare
Matthew B. Dwyer
Sebastian G. Elbaum

abstract: There has been significant interest in equipping programs with runtime checks aimed at detecting errors to improve fault detection during testing and in the field. Recent work in this area has studied methods for efficiently monitoring a program execution's conformance to path property specifications, e.g., such as those captured by a finite state automaton. These techniques show great promise, but their broad applicability is hampered by the fact that for certain combinations of programs and properties the overhead of checking can slow the program down by up to 3500%.


title: Homogeneous family sharing

authors:
Xin Qi
Andrew C. Myers

abstract: Recent work has introduced class sharing as a mechanism for adapting a family of related classes with new functionality. This paper introduces homogeneous family sharing, implemented in the J&h language, in which the sharing mechanism is lifted from class-level sharing to true family-level sharing. Compared to the original (heterogeneous) class sharing mechanism, homogeneous family sharing provides useful new functionality and substantially reduces the annotation burden on programmers by eliminating the need for masked types and sharing declarations. This is achieved through a new mechanism, shadow classes, which permit homogeneous sharing of all related classes in shared families. The new sharing mechanism has a straightforward semantics, which is formalized in the J&h calculus. The soundness of the J&h type system is proved. The J&h language is implemented as an extension to the J& language. To demonstrate the effectiveness of family sharing, the Polyglot compiler framework is ported to J&h.


title: Rubber ducks, nightmares, and unsaturated predicates: proto-scientific schemata are good for agile

authors:
Jenny Quillien
Dave West

abstract: Fine-grain case studies of scientific inquiry, lessons from linguistics on metaphoric thinking, the epistemology of Charles Sanders Peirce, recent work on architectural image-schemata, along with the computer world's own theorist, Peter Naur, all suggest that software developers (frequently dulled and desiccated from overdosing on 'Cartesian' methodologies) could benefit from imbibing a little 'mysticism' not the wave-your-hands woo-woo kind but the more ineffable hunch and gut side of human cognition. Scholarly publications in their final polished forms rarely admit that stories, jokes, eroticism, and dreams were the fertile seeds that germinated into 'serious' results. This essay looks to these 'closet' sources, non-reductionist, non-self conscious, metaphorical, aformal modes of thought as the salvation of a profession gone awry. It is notably proto-scientific image-schemata that retain our attention as a pragmatic tool for improving the fecundity of Agile methodology, at its roots, so to speak. The necessary context is provided by Peter Naur's fundamental insights about software development as 'theory building' coupled with an elaboration of the Agile concept of storytelling.


title: Concurrency by modularity: design patterns, a case in point

authors:
Hridesh Rajan
Steven M. Kautz
Wayne Rowcliffe

abstract: General purpose object-oriented programs typically aren't embarrassingly parallel. For these applications, finding enough concurrency remains a challenge in program design. To address this challenge, in the Panini project we are looking at reconciling concurrent program design goals with modular program design goals. The main idea is that if programmers improve the modularity of their programs they should get concurrency for free. In this work we describe one of our directions to reconcile these two goals by enhancing Gang-of-Four (GOF) object-oriented design patterns. GOF patterns are commonly used to improve the modularity of object-oriented software. These patterns describe strategies to decouple components in design space and specify how these components should interact. Our hypothesis is that if these patterns are enhanced to also decouple components in execution space applying them will concomitantly improve the design and potentially available concurrency in software systems. To evaluate our hypothesis we have studied all 23 GOF patterns. For 18 patterns out of 23, our hypothesis has held true. Another interesting preliminary result reported here is that for 17 out of these 18 studied patterns, concurrency and synchronization concerns were completely encapsulated in our concurrent design pattern framework.


title: What can the GC compute efficiently?: a language for heap assertions at GC time

authors:
Christoph Reichenbach
Neil Immerman
Yannis Smaragdakis
Edward Aftandilian
Samuel Z. Guyer

abstract: We present the DeAL language for heap assertions that are efficiently evaluated during garbage collection time. DeAL is a rich, declarative, logic-based language whose programs are guaranteed to be executable with good whole-heap locality, i.e., within a single traversal over every live object on the heap and a finite neighborhood around each object. As a result, evaluating DeAL programs incurs negligible cost: for simple assertion checking at each garbage collection, the end-to-end execution slowdown is below 2%. DeAL is integrated into Java as a VM extension and we demonstrate its efficiency and expressiveness with several applications and properties from the past literature.


title: Patterns and statistical analysis for understanding reduced resource computing

authors:
Martin C. Rinard
Henry Hoffmann
Sasa Misailovic
Stelios Sidiroglou

abstract: We present several general, broadly applicable mechanisms that enable computations to execute with reduced resources, typically at the cost of some loss in the accuracy of the result they produce.We identify several general computational patterns that interact well with these resource reduction mechanisms, present a concrete manifestation of these patterns in the form of simple model programs, perform simulationbased explorations of the quantitative consequences of applying these mechanisms to our model programs, and relate the model computations (and their interaction with the resource reduction mechanisms) to more complex benchmark applications drawn from a variety of fields.


title: Efficient modular glass box software model checking

authors:
Michael Roberson
Chandrasekhar Boyapati

abstract: Glass box software model checking incorporates novel techniques to identify similarities in the state space of a model checker and safely prune large numbers of redundant states without explicitly checking them. It is significantly more efficient than other software model checking approaches for checking certain kinds of programs and program properties.


title: Specifying and implementing refactorings

authors:
Max Schäfer
Oege de Moor

abstract: Modern IDEs for object-oriented languages like Java provide support for a basic set of simple automated refactorings whose behaviour is easy to describe intuitively. It is, however, surprisingly difficult to specify their behaviour in detail. In particular, the popular precondition-based approach tends to produce somewhat unwieldy descriptions if advanced features of the object language are taken into account. This has resulted in refactoring implementations that are complex, hard to understand, and even harder to maintain, yet these implementations themselves are the only precise specification of many refactorings. We have in past work advocated a different approach based on several complementary notions of dependencies that guide the implementation, and on the concept of microrefactorings that structure it. We show in this work that these concepts are powerful enough to provide high-level specifications of many of the refactorings implemented in Eclipse. These specifications are precise enough to serve as the basis of a clean-room reimplementation of these refactorings that is very compact, yet matches Eclipse's for features and outperforms it in terms of correctness.


title: MetaFJig: a meta-circular composition language for Java-like classes

authors:
Marco Servetto
Elena Zucca

abstract: We propose a Java-like language where class definitions are first class values and new classes can be derived from existing ones by exploiting the full power of the language itself, used on top of a small set of primitive composition operators, instead of using a fixed mechanism like inheritance.Hence, compilation requires to perform (meta-)reduction steps, by a process that we call compile-time execution. This approach differs from meta-programming techniques available in mainstream languages since it is meta-circular, hence programmers are not required to learn new syntax and idioms.Compile-time execution is guaranteed to be sound (not to get stuck) by a lightweight technique, where class composition errors are detected dynamically, and conventional typing errors are detected by interleaving typechecking with meta-reduction steps. This allows for a modular approach, that is, compile-time execution is defined, and can be implemented, on top of typechecking and execution of the underlying language. Moreover, programmers can handle errors due to composition operators.Besides soundness, our technique ensures an additional important property called meta-level soundness, that is, typing errors never originate from (meta-)code in already compiled programs.


title: Do I use the wrong definition?: DeFuse: definition-use invariants for detecting concurrency and sequential bugs

authors:
Yao Shi
Soyeon Park
Zuoning Yin
Shan Lu
Yuanyuan Zhou
Wenguang Chen
Weimin Zheng

abstract: Software bugs, such as concurrency, memory and semantic bugs, can significantly affect system reliability. Although much effort has been made to address this problem, there are still many bugs that cannot be detected, especially concurrency bugs due to the complexity of concurrent programs. Effective approaches for detecting these common bugs are therefore highly desired.


title: Programming with time: cyber-physical programming with impromptu

authors:
Andrew Sorensen
Henry J. Gardner

abstract: The act of computer programming is generally considered to be temporally removed from a computer program's execution. In this paper we discuss the idea of programming as an activity that takes place within the temporal bounds of a real-time computational process and its interactions with the physical world. We ground these ideas within the con- text of livecoding -- a live audiovisual performance practice. We then describe how the development of the programming environment "Impromptu" has addressed our ideas of programming with time and the notion of the programmer as an agent in a cyber-physical system.


title: Reasoning about multiple related abstractions with MultiStar

authors:
Stephan van Staden
Cristiano Calcagno

abstract: Encapsulated abstractions are fundamental in object-oriented programming. A single class may employ multiple abstractions to achieve its purpose. Such abstractions are often related and combined in disciplined ways. This paper explores ways to express, verify and rely on logical relationships between abstractions. It introduces two general specification mechanisms: export clauses for relating abstractions in individual classes, and axiom clauses for relating abstractions in a class and all its descendants. MultiStar, an automatic verification tool based on separation logic and abstract predicate families, implements these mechanisms in a multiple inheritance setting. Several verified examples illustrate MultiStar's underlying logic. To demonstrate the flexibility of our approach, we also used MultiStar to verify the core iterator hierarchy of a popular data structure library.


title: To achieve our highest goals, we must be willing to abandon them

abstract: Someday computer scientists may build systems of astronomical complexity that provide profound benefit to humanity. However, the question today is how such feats will ultimately be achieved. As is common with technology, present limitations to modern approaches challenge our imagination to search for new paradigms and organizing principles. To examine the prerequisites to achieving our most ambitious objectives, this talk will contemplate the implications of recent counterintuitive results from experiments with evolutionary algorithms that suggest that search (which is a metaphor for innovation in general) is sometimes most effective when it is not explicitly seeking an objective. In particular, through several experiments in interactive evolution and with an algorithm called "novelty search," a picture of innovation is emerging in which objectives can help to guide us one stepping-stone away from our present understanding, yet ultimately become handcuffs that also blind us to essential orthogonal discoveries on the road to long-term innovation. While the implications of these insights for reaching our highest goals are in part sobering, the silver lining is that much can be gained by liberating ourselves from the temptation to frame all our projects in terms of what they ultimately aim to achieve. Instead, with evidence in hand, we can exploit the structure of the unknown by orienting ourselves towards discovery and away from the shackles of mandated outcomes.


title: Back to the futures: incremental parallelization of existing sequential runtime systems

authors:
James Swaine
Kevin Tew
Peter A. Dinda
Robert Bruce Findler
Matthew Flatt

abstract: Many language implementations, particularly for high-level and scripting languages, are based on carefully honed runtime systems that have an internally sequential execution model. Adding support for parallelism in the usual form -- as threads that run arbitrary code in parallel -- would require a major revision or even a rewrite to add safe and efficient locking and communication. We describe an alternative approach to incremental parallelization of runtime systems. This approach can be applied inexpensively to many sequential runtime systems, and we demonstrate its effectiveness in the Racket runtime system and Parrot virtual machine. Our evaluation assesses both the performance benefits and the developer effort needed to implement our approach. We find that incremental parallelization can provide useful, scalable parallelism on commodity multicore processors at a fraction of the effort required to implement conventional parallel threads.


title: An input-centric paradigm for program dynamic optimizations

authors:
Kai Tian
Yunlian Jiang
Eddy Z. Zhang
Xipeng Shen

abstract: Accurately predicting program behaviors (e.g., locality, dependency, method calling frequency) is fundamental for program optimizations and runtime adaptations. Despite decades of remarkable progress, prior studies have not systematically exploited program inputs, a deciding factor for program behaviors.Triggered by the strong and predictive correlations between program inputs and behaviors that recent studies have uncovered, this work proposes to include program inputs into the focus of program behavior analysis, cultivating a new paradigm named input-centric program behavior analysis. This new approach consists of three components, forming a three-layer pyramid. At the base is program input characterization, a component for resolving the complexity in program raw inputs and the extraction of important features. In the middle is input-behavior modeling, a component for recognizing and modeling the correlations between characterized input features and program behaviors. These two components constitute input-centric program behavior analysis, which (ideally) is able to predict the large-scope behaviors of a program's execution as soon as the execution starts. The top layer of the pyramid is input-centric adaptation, which capitalizes on the novel opportunities that the first two components create to facilitate proactive adaptation for program optimizations.By centering on program inputs, the new approach resolves a proactivity-adaptivity dilemma inherent in previous techniques. Its benefits are demonstrated through proactive dynamic optimizations and version selection, yielding significant performance improvement on a set of Java and C programs.


title: Automatic atomic region identification in shared memory SPMD programs

authors:
Gautam Upadhyaya
Samuel P. Midkiff
Vijay S. Pai

abstract: This paper presents TransFinder, a compile-time tool that automatically determines which statements of an unsynchronized multithreaded program must be enclosed in atomic regions to enforce conflict-serializability. Unlike previous tools, TransFinder requires no programmer input (beyond the program) and is more efficient in both time and space.


title: Pinocchio: bringing reflection to life with first-class interpreters

authors:
Toon Verwaest
Camillo Bruni
David Gurtner
Adrian Lienhard
Oscar Nierstrasz

abstract: To support development tools like debuggers, runtime systems need to provide a meta-programming interface to alter their semantics and access internal data. Reflective capabilities are typically fixed by the Virtual Machine (VM). Unanticipated reflective features must either be simulated by complex program transformations, or they require the development of a specially tailored VM. We propose a novel approach to behavioral reflection that eliminates the barrier between applications and the VM by manipulating an explicit tower of first-class interpreters. Pinocchio is a proof-of-concept implementation of our approach which enables radical changes to the interpretation of programs by explicitly instantiating subclasses of the base interpreter. We illustrate the design of Pinocchio through non-trivial examples that extend runtime semantics to support debugging, parallel debugging, and back-in-time object-flow debugging. Although performance is not yet addressed, we also discuss numerous opportunities for optimization, which we believe will lead to a practical approach to behavioral reflection.


title: Cross-language, type-safe, and transparent object sharing for co-located managed runtimes

authors:
Michal Wegiel
Chandra Krintz

abstract: As software becomes increasingly complex and difficult to analyze, it is more and more common for developers to use high-level, type-safe, object-oriented (OO) programming languages and to architect systems that comprise multiple components. Different components are often implemented in different programming languages. In state-of-the-art multicomponent, multi-language systems, cross-component communication relies on remote procedure calls (RPC) and message passing. As components are increasingly co-located on the same physical machine to ensure high utilization of multi-core systems, there is a growing potential for using shared memory for cross-language cross-runtime communication.


title: Composable specifications for structured shared-memory communication

authors:
Benjamin P. Wood
Adrian Sampson
Luis Ceze
Dan Grossman

abstract: In this paper we propose a communication-centric approach to specifying and checking how multithreaded programs use shared memory to perform inter-thread communication. Our approach complements past efforts for improving the safety of multithreaded programs such as race detection and atomicity checking. Unlike prior work, we focus on what pieces of code are allowed to communicate with one another, as opposed to declaring what data items are shared or what code blocks should be atomic. We develop a language that supports composable specifications at multiple levels of abstraction and that allows libraries to specify whether or not shared-memory communication is exposed to clients. The precise meaning of a specification is given with a formal semantics we present. We have developed a dynamic-analysis tool for Java that observes program execution to see if it obeys a specification. We report results for using the tool on several benchmark programs to which we added specifications, concluding that our approach matches the modular structure of multithreaded applications and that our tool is performant enough for use in development and testing.


title: Ownership and immutability in generic Java

authors:
Yoav Zibin
Alex Potanin
Paley Li
Mahmood Ali
Michael D. Ernst

abstract: The Java language lacks the important notions of ownership (an object owns its representation to prevent unwanted aliasing) and immutability (the division into mutable, immutable, and readonly data and references). Programmers are prone to design errors, such as representation exposure or violation of immutability contracts. This paper presents Ownership Immutability Generic Java (OIGJ), a backward-compatible purely-static language extension supporting ownership and immutability. We formally defined a core calculus for OIGJ, based on Featherweight Java, and proved it sound. We also implemented OIGJ and performed case studies on 33,000 lines of code.


title: Oracle scheduling: controlling granularity in implicitly parallel languages

authors:
Umut A. Acar
Arthur Charguéraud
Mike Rainey

abstract: A classic problem in parallel computing is determining whether to execute a task in parallel or sequentially. If small tasks are executed in parallel, the task-creation overheads can be overwhelming. If large tasks are executed sequentially, processors may spin idle. This granularity problem, however well known, is not well understood: broadly applicable solutions remain elusive.


title: Flow-sensitive type recovery in linear-log time

authors:
Michael D. Adams
Andrew W. Keep
Jan Midtgaard
Matthew Might
Arun Chauhan
R. Kent Dybvig

abstract: The flexibility of dynamically typed languages such as JavaScript, Python, Ruby, and Scheme comes at the cost of run-time type checks. Some of these checks can be eliminated via control-flow analysis. However, traditional control-flow analysis (CFA) is not ideal for this task as it ignores flow-sensitive information that can be gained from dynamic type predicates, such as JavaScript's 'instanceof' and Scheme's 'pair?', and from type-restricted operators, such as Scheme's 'car'. Yet, adding flow-sensitivity to a traditional CFA worsens the already significant compile-time cost of traditional CFA. This makes it unsuitable for use in just-in-time compilers. In response, we have developed a fast, flow-sensitive type-recovery algorithm based on the linear-time, flow-insensitive sub-0CFA. The algorithm has been implemented as an experimental optimization for the commercial Chez Scheme compiler, where it has proven to be effective, justifying the elimination of about 60% of run-time type checks in a large set of benchmarks. The algorithm processes on average over 100,000 lines of code per second and scales well asymptotically, running in only O(n log n) time. We achieve this compile-time performance and scalability through a novel combination of data structures and algorithms.


title: Asynchronous assertions

authors:
Edward Aftandilian
Samuel Z. Guyer
Martin T. Vechev
Eran Yahav

abstract: Assertions are a familiar and widely used bug detection technique. Traditional assertion checking, however, is performed synchronously, imposing its full cost on the runtime of the program. As a result, many useful kinds of checks, such as data structure invariants and heap analyses, are impractical because they lead to extreme slowdowns. We present a solution that decouples assertion evaluation from program execution: assertions are checked asynchronously by separate checking threads while the program continues to execute. Our technique guarantees that asynchronous evaluation always produces the same result as synchronous evaluation, even if the program concurrently modifies the program state. The checking threads evaluate each assertion on a consistent snapshot of the program state as it existed at the moment the assertion started.


title: Type checking modular multiple dispatch with parametric polymorphism and multiple inheritance

authors:
Eric E. Allen
Justin Hilburn
Scott Kilpatrick
Victor Luchangco
Sukyoung Ryu
David Chase
Guy L. Steele Jr.

abstract: In previous work, we presented rules for defining overloaded functions that ensure type safety under symmetric multiple dispatch in an object-oriented language with multiple inheritance, and we showed how to check these rules without requiring the entire type hierarchy to be known, thus supporting modularity and extensibility. In this work, we extend these rules to a language that supports parametric polymorphism on both classes and functions.


title: Composable, nestable, pessimistic atomic statements

authors:
Zachary R. Anderson
David Gay

abstract: In this paper we introduce a new method for pessimistically implementing composable, nestable atomic statements. Our mechanism, called shelters, is inspired by the synchronization strategy used in the Jade programming language. Unlike previous lock-based pessimistic approaches, our mechanism does not require a whole-program analysis that computes a global lock order. Further, this mechanism frees us to implement several optimizations, impossible with automatically inserted locks, that are necessary for scaling on recent multi-core systems. Additionally we show how our basic mechanism can be extended to support both open- and closed-nesting of atomic statements, something that, to our knowledge, has not yet been implemented fully-pessimistically in this context. Unlike optimistic, transactional-memory-based approaches, programmers using our mechanism do not have to write compensating actions for open-nesting, or worry about the possibly awkward semantics and performance impact of aborted transactions.


title: Virtual values for language extension

authors:
Thomas H. Austin
Tim Disney
Cormac Flanagan

abstract: This paper focuses on extensibility, the ability of a programmer using a particular language to extend the expressiveness of that language. This paper explores how to provide an interesting notion of extensibility by virtualizing the interface between code and data. A virtual value is a special value that supports behavioral intercession. When a primitive operation is applied to a virtual value, it invokes a trap on that virtual value. A virtual value contains multiple traps, each of which is a user-defined function that describes how that operation should behave on that value. This paper formalizes the semantics of virtual values, and shows how they enable the definition of a variety of language extensions, including additional numeric types; delayed evaluation; taint tracking; contracts; revokable membranes; and units of measure. We report on our experience implementing virtual values for Javascript within an extension for the Firefox browser.


title: Two for the price of one: a model for parallel and incremental computation

authors:
Sebastian Burckhardt
Daan Leijen
Caitlin Sadowski
Jaeheon Yi
Thomas Ball

abstract: Parallel or incremental versions of an algorithm can significantly outperform their counterparts, but are often difficult to develop. Programming models that provide appropriate abstractions to decompose data and tasks can simplify parallelization. We show in this work that the same abstractions can enable both parallel and incremental execution. We present a novel algorithm for parallel self-adjusting computation. This algorithm extends a deterministic parallel programming model (concurrent revisions) with support for recording and repeating computations. On record, we construct a dynamic dependence graph of the parallel computation. On repeat, we reexecute only parts whose dependencies have changed.


title: Benefits and barriers of user evaluation in software engineering research

authors:
Raymond P. L. Buse
Caitlin Sadowski
Westley Weimer

abstract: In this paper, we identify trends about, benefits from, and barriers to performing user evaluations in software engineering research. From a corpus of over 3,000 papers spanning ten years, we report on various subtypes of user evaluations (e.g., coding tasks vs. questionnaires) and relate user evaluations to paper topics (e.g., debugging vs. technology transfer). We identify the external measures of impact, such as best paper awards and citation counts, that are correlated with the presence of user evaluations. We complement this with a survey of over 100 researchers from over 40 different universities and labs in which we identify a set of perceived barriers to performing user evaluations.