Methodology of
Programming Systems

Joel Jakubovic, Tomas Petricek, Jonathan Edwards
tomas@tomasp.net | @tomaspetricek

Programming Languages

We know how to study this!

Programming Systems

But how do we study this?

Theory bias

Establishing computer science as a legitimate theoretical discipline was (...) an essential component in the professionalization agenda

Within the status hierarchy of the university (...) theory ranked higher than practice, and was therefore desirable for its own sake.

It provided a means of distinguishing the competent professional from
the mere technician.

Communities

Communities looking at programming systems


Programming experience and liveness (PX, LIVE)

Interaction and user experience (UIST, VL/HCC)

Formal models of programming systems (POPL)

History and philosophy of programming (HaPoP)



Methodologies

Different practices for evaluation


We prove safety of our tiny formal model...

We implemented a system and it runs fast.

Study shows users can complete tasks 10% faster...

We describe this interesting past influential system.

Here is a new system we built. Behold!



The missing component

Capture essential characteristics of system design!

How the system makes you think

How the system makes you interact

Possible answers

Evaluating programming systems design

Formalism
done right?

Program as a sequence of interactions with the environment

More realistic, but narrow technical focus


Videos and screencasts?

Walkthrough showing concrete use case of working with a system

How to generalize to get reusable knowledge?


Interactive essays?

Guided interaction, allowing the reader to explore independently

Interaction focus systems, but what makes it rigorous?


Complementary science?

Recover interesting ideas about past programming systems

Is a limited simulation accurate enough?


Technical Dimensions of Programming Systems?