Because software engineering often times distributes work across multiple people, a fundamental challenge in software engineering is ensuring that everyone on a team has the same understanding of what is being built and why. In the seminal book “The Mythical Man Month”, Fred Brooks argued that good software needs to have conceptual integrity, both in how it is designed, but also how it is implemented (Brooks 1995). This is the idea that whatever vision of what is being built must stay intact, even as the building of it gets distributed to multiple people. When multiple people are responsible for implementing a single coherent idea, how can they ensure they all build the same idea?
++ Because software engineering often times distributes work across multiple people, a fundamental challenge in software engineering is ensuring that everyone on a team has the same understanding of what is being built and why. + In the seminal book “The Mythical Man Month”, Fred Brooks argued that good software needs to have conceptual integrity, both in how it is designed, but also how it is implemented (Brooks 1995). + This is the idea that whatever vision of what is being built must stay intact, even as the building of it gets distributed to multiple people. + When multiple people are responsible for implementing a single coherent idea, how can they ensure they all build the same idea? +
-The solution is effective communication. When communication is poor and teams are disconnected, software defects are the result (Bettenburg & Hassan 2013). The social relationships in a team also play a large role in structuring how projects evolve (Zhou & Mockus 2011). Perhaps the most notable theory underlying these ideas is Conway's Law (Conway 1968), which argues that any designed system—software included—will reflect the social relationships behind its design. For example, look at any online banking website: the way the application is designed, how information is organized, the terminology that is used, and even the visual design is a reflection of how the teams inside that bank are organized and socially connected. Put simply, Conway observed that the architectural organization of the product resembled the organizational hierarchy of the teams that produced it.
++ The solution is effective communication. + As some events in industry have shown, communication requires empathy and teamwork. + When communication is poor, teams become disconnected and produce software defects (Bettenburg & Hassan 2013). + Therefore, achieving effective communication practices is paramount. +
+ ++ It turns out, however, that communication plays such a powerful role in software projects that it even shapes how projects unfold. + Perhaps the most notable theory about the effect of communication is Conway's Law (Conway 1968). + This theory argues that any designed system—software included—will reflect the communication structures involved in producing it. + For example, think back to any course project where you divided the work into chunks and tried to combine them together into a final report at the end. + The report and it's structure probably mirrored the fact that several distinct people worked on each section of the report, rather than sounding like a single coherent voice. + The same things happen in software: if the team writing error messages for a website isn't talking to the team presenting them, you're probably going to get a lot of error messages that aren't so clear, may not fit on screen, and may not be phrased using the terminology of the rest of the site. + On the other hand, if those two teams meet regularly to design the error mesages together, communicating their shared knowledge, they might produce a seamless, coherent experience. + Not only does software follow this law when a project is created, they also follow this law as projects evolve over time (Zhou & Mockus 2011). +
Because communication is so central, software engineers are constantly seeking information to further their work, going to their coworkers' desks, emailing them, chatting via messaging platforms, and even using social media (Ko et al. 2007). Some of the information that developers are seeking is easier to find than others. For example, in the study I just cited, it was pretty trivial to find information about how wrote a line of code or whether a build was done, but when the information they needed resided in someone else's head (e.g., why a particular line of code was written), it was slow or often impossible to retrieve it. Sometimes it's not even possible to find out who has the information. Researchers have investigated tools for trying to quantify expertise by automatically analyzing the code that developers have written, building platforms to help developers search for other developers who might know what they need to know (Mockus & Herbsleb 2002, Begel et al. 2010).
@@ -38,7 +59,15 @@Communication isn't just about transmitting information; it's also about relationships and identity. For example, the dominant culture of many software engineering work environments—and even the perceived culture—is one that can deter many people from even pursuing careers in computer science. Modern work environments are still dominated by men, who speak loudly, out of turn, and disrespectfully, with some even bordering on sexual harassment. Similarly, software developers often have to work with people in other domains such as artists, content developers, data scientists, design researchers, designers, electrical engineers, mechanical engineers, product planners, program managers, and service engineers. One study found that developers' cross-disciplinary collaborations with people in these other domains required open-mindedness about the input of others, proactively informing everyone about code-related constraints, and ultimately seeing the broader picture of how pieces from different disciplines fit together; when developers didn't do these things, collaborations failed, and therefore projects failed (Li et al. 2017). These are not the conditions for trusting, effective communication.
-When communication is effective, it still takes time. One of the key strategies for reducing the amount of communication necessary is knowledge sharing tools, which broadly refers to any information system that stores facts that developers would normally have to retrieve from a person. By storing them in a database and making them easy to search, teams can avoid interruptions. The most common knowledge sharing tools in software teams are issue trackers, which are often at the center of communication not only between developers, but also with every other part of a software organization (Bertram et al. 2010). Community portals, such as GitHub pages or Slack teams, can also be effective ways of sharing documents and archiving decisions (Treude & Storey 2011). Perhaps the most popular knowledge sharing tool in software engineering today is Stack Overflow, which archives facts about programming language and API usage.
++ When communication is effective, it still takes time. + One of the key strategies for reducing the amount of communication necessary is knowledge sharing tools, which broadly refers to any information system that stores facts that developers would normally have to retrieve from a person. + By storing them in a database and making them easy to search, teams can avoid interruptions. + The most common knowledge sharing tools in software teams are issue trackers, which are often at the center of communication not only between developers, but also with every other part of a software organization (Bertram et al. 2010). + Community portals, such as GitHub pages or Slack teams, can also be effective ways of sharing documents and archiving decisions (Treude & Storey 2011). + Perhaps the most popular knowledge sharing tool in software engineering today is Stack Overflow, which archives facts about programming language and API usage. + Such sites, while they can be great resources, have the same problems as many media, such as gender bias that prevent contributions from women from being rewarded as highly as contributions from men (May et al. 2019). +
Because all of this knowledge is so critical to progress, when developers leave an organization and haven't archived their knowledge somewhere, it can be quite disruptive to progress. Organizations often have single points of failure, in which a single developer may be critical to a team's ability to maintain and enhance a software product (Rigby et al. 2016). When newcomers join a team and lack the right knowledge, they introduce defects (Foucault et al. 2015). Some companies try to mitigate this by rotating developers between projects, “cross-training” them to ensure that the necessary knowledge to maintain a project is distributed across multiple engineers.
@@ -61,6 +90,7 @@Andrew J. Ko, Robert DeLine, and Gina Venolia. 2007. Information Needs in Collocated Software Development Teams. In Proceedings of the 29th international conference on Software Engineering (ICSE '07). IEEE Computer Society, Washington, DC, USA, 344-353.
Li, P. L., Ko, A. J., & Begel, A. (2017, May). Cross-disciplinary perspectives on collaborations with software engineers. In Proceedings of the 10th International Workshop on Cooperative and Human Aspects of Software Engineering (pp. 2-8).
Mark, G., Gudith, D., & Klocke, U. (2008, April). The cost of interrupted work: more speed and stress. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 107-110).
+May, A., Wachs, J., & Hannák, A. (2019). Gender differences in participation and reward on Stack Overflow. Empirical Software Engineering, 1-23.
Audris Mockus. 2010. Organizational volatility and its effects on software defects. In Proceedings of the eighteenth ACM SIGSOFT international symposium on Foundations of software engineering (FSE '10). ACM, New York, NY, USA, 117-126.
Audris Mockus and James D. Herbsleb. 2002. Expertise browser: a quantitative approach to identifying expertise. In Proceedings of the 24th International Conference on Software Engineering (ICSE '02). ACM, New York, NY, USA, 503-512.
Perlow, L. A. (1999). The time famine: Toward a sociology of work time. Administrative science quarterly, 44(1), 57-81.
diff --git a/productivity.html b/productivity.html index 9422c4f..eb9871b 100644 --- a/productivity.html +++ b/productivity.html @@ -30,11 +30,31 @@When we think of productivity, we usually have a vague concept of a rate of work per unit time. Where it gets tricky is in defining "work". On an individual level, work can be easier to define, because developers often have specific concrete tasks that they're assigned. But until they're not, it's not really easy to define progress (well, it's not that easy to define "done" sometimes either, but that's a topic for a later chapter). When you start considering work at the scale of a team or an organization, productivity gets even harder to define, since an individual's productivity might be increased by ignoring every critical request from a teammate, harming the team's overall productivity.
-Despite the challenge in defining productivity, there are numerous factors that affect productivity. For example, at the individual level, having the right tools can result in an order of magnitude difference in speed at accomplishing a task. One study I ran found that developers using the Eclipse IDE spent a third of their time just physically navigating between source files (Ko et al. 2005). With the right navigation aids, developers could be writing code and fixing bugs 30% faster. In fact, some tools like Mylyn automatically bring relevant code to the developer rather than making them navigate to it, greatly increasing the speed which with developers can accomplish a task (Kersten & Murphy 2006). Long gone are the days when developers should be using bare command lines and text editors to write code: IDEs can and do greatly increase productivity when used and configured with speed in mind.
++ Despite the challenge in defining productivity, there are numerous factors that affect productivity. + For example, at the individual level, having the right tools can result in an order of magnitude difference in speed at accomplishing a task. + One study I ran found that developers using the Eclipse IDE spent a third of their time just physically navigating between source files (Ko et al. 2005). + With the right navigation aids, developers could be writing code and fixing bugs 30% faster. + In fact, some tools like Mylyn automatically bring relevant code to the developer rather than making them navigate to it, greatly increasing the speed which with developers can accomplish a task (Kersten & Murphy 2006). + Long gone are the days when developers should be using bare command lines and text editors to write code: IDEs can and do greatly increase productivity when used and configured with speed in mind. +
+ ++ Of course, individual productivity is about more than just tools. + Studies of workplace productivity show that developers have highly fragmented days, interrupted by meetings, emails, coding, and non-work distractions (Meyer et al. 2017). + These interruptions are often viewed negatively from an individual perspective, but may be highly valuable from a team and organizational perspective. + And then, productivity is not just about skills to manage time, but also many other skills that shape developer expertise, including skills in designing architectures, debugging, testing, programming languages, etc. (Baltes et al. 2018). +
That said, productivity is not just about individual developers. Because communication is a key part of team productivity, an individual's productivity is as much determined by their ability to collaborate and communicate with other developers. In a study spanning dozens of interviews with senior software engineers, Li et al. found that the majority of critical attributes for software engineering skill (productivity included) concerned their interpersonal skills, their communication skills, and their ability to be resourceful within their organization (Li et al. 2015). Similarly, LaToza et al. found that the primary bottleneck in productivity was communication with teammates, primarily because waiting for replies was slower than just looking something up (LaToza et al. 2006). Of course, looking something up has its own problems. While StackOverflow is an incredible resource for missing documentation (Mamykina et al. 2001), it also is full of all kinds of misleading and incorrect information contributed by developers without sufficient expertise to answer questions (Barua et la. 2014). Finally, because communication is such a critical part of retrieving information, adding more developers to a team has surprising effects. One study found that adding people to a team slowly enough to allow them to onboard effectively could reduce defects, but adding them too fast led to increases in defects (Meneely et al. 2011).
-Another dimension of productivity is learning. Great engineers are resourceful, quick learners (Li et al. 2015); new engineers must be even more resourceful, even though their instincts are often to hide their lack of expertise from exactly the people they need help from (Begel & Simon 2008). Experienced developers know that learning is important and now rely heavily on social media such as Twitter to follow industry changes, build learning relationships, and discover new concepts and platforms to learn (Singer et al. 2012).
++ Another dimension of productivity is learning. + Great engineers are resourceful, quick learners (Li et al. 2015). + New engineers must be even more resourceful, even though their instincts are often to hide their lack of expertise from exactly the people they need help from (Begel & Simon 2008). + Experienced developers know that learning is important and now rely heavily on social media such as Twitter to follow industry changes, build learning relationships, and discover new concepts and platforms to learn (Singer et al. 2012). + And, of course, developers now rely heavily on web search to fill in inevitable gaps in their knowledge about APIs, error messages, and myriad other details about languages and platforms (Xia et al. 2017). +
Unfortunately, learning is no easy task. One of my earliest studies as a researcher investigated the barriers to learning new programming languages and systems, finding six distinct types of content that are challenging (Ko & Myers 2004). To use a programming platform successfully, people need to overcome design barriers, which are the abstract computational problems that must be solved, independent of the languages and APIs. People need to overcome selection barriers, which involve finding the right abstractions or APIs to achieve the design they have identified. People need to overcome use and coordination barriers, which involve operating and coordinating different parts of a language or API together to achieve novel functionality. People need to overcome comprehension barriers, which involve knowing what can go wrong when using part of a language or API. And finally, people need to overcome information barriers, which are posed by the limited ability of tools to inspect a program's behavior at runtime during debugging. Every single one of these barriers has its own challenges, and developers encounter them every time they are learning a new platform, regardless of how much expertise they have.
@@ -56,6 +76,11 @@One could imagine using these concepts to refine processes and practices in a team, helping both developers and managers be more aware of sources of waste that harm productivity.
++ These widely varying conceptions of productivity reveal that programming in a software engineering context is about far more than just writing a lot of code. + It's about coordinating productively with a team, synchronizing your work with an organizations goals, and most importantly, reflecting on ways to change work to achieve those goals more effectively. +
+Barua, A., Thomas, S. W., & Hassan, A. E. (2014). What are developers talking about? an analysis of topics and trends in stack overflow. Empirical Software Engineering, 19(3), 619-654.
+Baltes, S., & Diehl, S. (2018, October). Towards a theory of software development expertise. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 187-200). ACM.
Begel, A., & Simon, B. (2008, September). Novice software developers, all over again. In Proceedings of the Fourth international Workshop on Computing Education Research (pp. 3-14). ACM.
Casey Casalnuovo, Bogdan Vasilescu, Premkumar Devanbu, and Vladimir Filkov. 2015. Developer onboarding in GitHub: the role of prior social links and language experience. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). ACM, New York, NY, USA, 817-828.
Jan Chong and Tom Hurlbutt. 2007. The Social Dynamics of Pair Programming. In Proceedings of the 29th international conference on Software Engineering (ICSE '07). IEEE Computer Society, Washington, DC, USA, 354-363.
@@ -76,11 +102,12 @@Thomas D. LaToza, Gina Venolia, and Robert DeLine. 2006. Maintaining mental models: a study of developer work habits. In Proceedings of the 28th international conference on Software engineering (ICSE '06). ACM, New York, NY, USA, 492-501.
Mamykina, L., Manoim, B., Mittal, M., Hripcsak, G., & Hartmann, B. (2011, May). Design lessons from the fastest q&a site in the west. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 2857-2866).
Andrew Meneely, Pete Rotella, and Laurie Williams. 2011. Does adding manpower also affect quality? An empirical, longitudinal analysis. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering (ESEC/FSE '11). ACM, New York, NY, USA, 81-90.
-André N. Meyer, Thomas Fritz, Gail C. Murphy, and Thomas Zimmermann. 2014. Software developers' perceptions of productivity. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). ACM, New York, NY, USA, 19-29.
+Meyer, A. N., Barton, L. E., Murphy, G. C., Zimmermann, T., & Fritz, T. (2017). The work life of developers: Activities, switches and perceived productivity. IEEE Transactions on Software Engineering, 43(12), 1178-1193.
Sedano, T., Ralph, P., & Péraire, C. (2017, May). Software development waste. In Proceedings of the 39th International Conference on Software Engineering (pp. 130-140). IEEE Press.
Leif Singer, Fernando Figueira Filho, and Margaret-Anne Storey. 2014. Software engineering at the speed of light: how developers stay current using twitter. In Proceedings of the 36th International Conference on Software Engineering (ICSE 2014). ACM, New York, NY, USA, 211-221.
Jeffrey Stylos and Brad A. Myers. 2008. The implications of method placement on API learnability. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering (SIGSOFT '08/FSE-16). ACM, New York, NY, USA, 105-112.
J. Vosburgh, B. Curtis, R. Wolverton, B. Albert, H. Malec, S. Hoben, and Y. Liu. 1984. Productivity factors and programming environments. In Proceedings of the 7th international conference on Software engineering (ICSE '84). IEEE Press, Piscataway, NJ, USA, 143-152.
+Xia, X., Bao, L., Lo, D., Kochhar, P. S., Hassan, A. E., & Xing, Z. (2017). What do developers search for on the web? Empirical Software Engineering, 22(6), 3149-3185.