OOPSLA 2004 CFP: Component and Middleware Performance Workshop

[Petr Tuma <petr.tuma@xxxxxxxxxxx>]

         (Apologies if you receive multiple copies of this post.)


               CALL FOR PAPERS : OOPSLA 2004 Workshop
                Component and Middleware Performance

     Deadline September 2, 2004 - Conference October 24-28, 2004
       Published in special issue of Studia Informatica journal
           Vancouver Convention & Exhibition Centre, Canada
                http://nenya.ms.mff.cuni.cz/workshop


OVERVIEW: The goal of the workshop is to help advance the current 
practice of gathering performance characteristics of component systems 
and middleware implementations through benchmarking. Positions are 
solicited especially from people with previous or impending benchmarking 
experience.

The participants of the workshop will identify requirements and 
obstacles of component and middleware benchmarking and form a position 
on issues such as designing a framework that would allow the design, 
running and evaluating a diverse range of benchmarks over a diverse 
range of platforms, designing benchmark criteria that would allow for a 
meaningful comparison of results collected over different platforms, 
designing a framework suitable for regular regression testing, or 
providing means to verify the benchmarking results and their 
applicability to specific usage situations.


SUBMISSIONS: The attendees are invited to submit a statement of 
interest, a position paper or an experience report. The submitted 
materials should be reasonably concise. The organizing committee will 
disseminate all submissions to the workshop attendees prior to the 
workshop and select position papers and experience reports to be 
presented at the beginning of the workshop. The position papers and 
experience reports selected for presentation will be published in a 
special issue of the Studia Informatica journal on components and 
performance, scheduled for publication in Spring 2005. A joint position 
of the attendees will be considered for publication.

The topics are envisioned to include, but not be limited, to:

Current practice
 - Experience with existing benchmarks and benchmarking frameworks
   (ECperf, JMeter, Load Runner, OpenCB, OpenSTA, RUBiS and others).
 - Existing component and middleware benchmarking initiatives and
   standardization efforts (JCP, SPEC, TSS and others).

Issues concerning benchmarks
 - Impact and characteristics of various object oriented designs in
   benchmarks.
 - Characterizing requirements on platform performance.
 - Characterizing workload and background load.
 - Platform independent benchmark specification.

Issues concerning benchmarking frameworks
 - Designing frameworks for a diverse range of benchmarks over a
   diverse range of platforms.
 - Designing frameworks for standalone benchmarking of components.
 - Designing framework for regular regression testing.

Issues concerning benchmarking applications
 - Use of benchmarking results for performance modeling.
 - Meaningful comparison of results collected over different
   platforms.
 - Guidelines for carrying out performance and scalability
   evaluation studies.
 - Use of benchmarking results for predictable assembly of
   components and middleware.
 - Means of verifying the benchmarking results and their
   applicability to specific usage situations.

The statements of interest, position papers and experience reports 
should be mailed to the organizing committee at 
oopsla-workshop@xxxxxxxxxxxxxxxxxxxxx Any of the DOC, PDF, PS or RTF 
formats are accepted, for formatting instructions please see the Studia 
Informatica journal at http://www.studia-informatica.org.


IMPORTANT DATES:
  Submission deadline for early registration:   September 2, 2004
  Notification of selection for presentation:   September 9, 2004
  OOPSLA early registration deadline:           September 16, 2004
  OOPSLA conference:                            October 24-28, 2004

CONTACT:
  Workshop web page                    http://nenya.ms.mff.cuni.cz/workshop
  Organizing and program committees    oopsla-workshop@xxxxxxxxxxxxxxxxxxxx

Organizing committee
 - Paul Brebner, University College London, United Kingdom
 - Emmanuel Cecchet, ObjectWeb?, INRIA Rhone-Alpes, France
 - Petr Tuma, Charles University, Czech Republic

Program committee
 - Paul Brebner, University College London, United Kingdom
 - Emmanuel Cecchet, ObjectWeb?, INRIA Rhone-Alpes, France
 - Patrick Lardieri, Lockheed Martin Advanced Technology Labs,
   United States
 - John Murphy, Dublin City University, Ireland
 - Balachandran Natarajan, Vanderbilt University, United States
 - Gautam G. Thaker, Lockheed Martin Advanced Technology Labs,
   United States
 - Petr Tuma, Charles University, Czech Republic