Summary

CESI2017 is about “conducting empirical studies in industrial settings”. We aim at identifying, debating about, and mitigating the barriers challenging the design and execution of empirical studies in industrial settings. In the past workshops, we also aimed at improved understanding of the emergence of industrial-strength empirical results and the critical characteristics of the research methods needed to yield those results as well as aggregating individual studies’ results towards practical, evidence-based guidelines.

This year, we would like to give a special emphasis on building and managing big data systems and the use and benefit of empirical studies in this context. Big data systems bring new types of software engineering challenges such as test optimization, planning for new requirements and/ or enhancements, optimization of code/ algorithms, decentralization of development tasks, the role of software analytics, etc.

As of today, most of the research findings of empirical studies are isolated in papers published by specific research groups. However, it is widely accepted that replication studies are important to build a body of knowledge in empirical software engineering. They enable transfer of novel approaches and findings into different settings and contexts. We also believe that: (i) improved understanding of challenges in conducting empirical studies in industry, (ii) considering these challenges in the design of experiments, and (iii) replicating empirical studies in different settings, would lead to actionable outcomes. Thus, in the proposed workshop, we are also interested in determining the kind of follow-on actions (recommendations) that are taken upon the results of empirical studies. Such recommendations have become even more important in the era of managing big data/ data-intensive systems that designing and managing such systems require empirical analysis in the field. The following topics exemplify the envisaged theme:

  • Impact of study results on industrial practice, i.e., their utility in the context of big data systems
  • Replication or families of studies in different industrial settings and in the context of big data systems
  • Issues in sharing datasets.
  • Reconciling researchers’ needs for “clean” and complete data and information with practitioners’ situations such as missing data, privacy issues, preservation of reputation.
  • How to choose relevant research questions?
  • Communication between researchers and practitioners.
  • Stakeholder involvement in empirical studies.
  • Dealing with threats in organizational settings.
  • Interpreting results in industrial contexts.
  • Generalizing the findings from case studies.
  • Impact of industrial settings on the design of, and on conducting empirical studies.
  • Understanding failures and successes: lessons learned.
  • Quantitative versus qualitative approaches.
  • Dealing with perceptions and biases.
  • Aggregating results from individual studies.
  • Use of (big) data analytics approaches
  • Application area digitization