Your Development Team's 6 Biggest Challenges

A few Rock Stars of Development have become popularized icons. In movies and TV these factual, and often fictionalized developers, have become heroes and anti-heroes, but in the real world, the other tens of millions of developers face the every day challenges in the trenches we call work.

In television shows like Mr. Robot and Person of Interest, the software developer like Elliot Aldersen and Harold Finch have become both hero and anti-hero saving and/or destroying the world. In the real world, former hackers like Steve Jobs, Bill Gates and Mark Zuckerberg are popular icons that even non-techies know and (sometimes) admire. By "hacker" I don't mean the criminal type, but rather the original use of the word, as a maverick or revolutionary from the outside. On his or her own, or through the aid of a small group, they're driven to change the world through their passion for computer technology. Of course, in a television series, there is much dramatic license to keep us watching and tech icons tend to be more famous for building their companies rather than their code. 

True software development icons are typically ones very few of us know - like Dennis Ritchie. As the creator of the C programming language and co-creator of UNIX, his work provides the underpinning of the entire computer industry. Windows, IOS, Java and the entire internet are either directly built on top of his creations or through programming that has evolved from the original C. His impact may be far greater than Jobs, Gates and Zuckerberg combined but it mostly remains under the radar. Based on what I've been able to research about Ritchie, it would seem he was just fine with the way things played out.

So if the Michael Jordan of software development is an unknown and unsung hero, then where does that leave the 11 million or more working software developers? For most of those 11 million the goal probably isn't to be a hero hacker, Tech Icon or even the next Dennis Ritchie. Their goals may be more mundane, but equally worthy - a project that will help colleagues become more productive, software that will help their company bring in more customers, developing a new programming language to help advance their career, or just getting through a stressful day in order to get home.

Survey Says
Stack Overflow conducted a survey of 56,033 developers in 173 countries. The top 6 challenges most cited by the respondents were:
  1. Unrealistic expectations
  2. Poor documentation
  3. Unspecific requirements
  4. Inefficient development processes
  5. Fragile code base
  6. Changing requirements
 Unrealistic Expectations
Almost all companies are going through a digital transformation because today every aspect of business is dependent on digital technology. Big data, analytics, machine learning, IoT, cloud and mobile computing all rely on software developers to bring them into being. The exponential growth and improvement of technology also means that everything is in it's early stages when it's being developed. Waiting for the technology to mature before implementing probably means something else has replaced it and provides the better competitive advantage. Add in the lack of complete understanding of the technology from leadership and management, as well as the belief that the technology will do much more than it should, often creates a situation where expectations far exceed actual experience.
 Unspecific & Changing Requirements 

Most projects begin with requirements gathering and proceed through sequential steps such as design, development, testing, etc. Even on small projects it's likely that new or changed requirements come at any step in the process and even post production deploy.  We just are not able to fully visualize what we want before we get our hands and eyes on the application, not to mention the external changes or shifts in project variables that can also drive changing requirements.
Internal Factors

The remaining three challenges, inefficient development processes, fragile code base and poor documentation, are internal where the previous three were external. However, they're all inter-related. The requirements issue can drive the inefficiency in the development process. As most projects may still follow the waterfall methodology, the requirements may change after a large investment in coding time. Those changes may drive going back and reworking or even redesigning the initial coding work. 

Poor documentation is referencing the code itself rather than external user type of documentation. The objective is to have highly readable code. There are a wide variety of thoughts on this from the development community. Some feel that code should be written as "really obvious code" (ROC), where any other developer will have a complete understanding from the code itself and further comments are a waste of time. Others feel a certain amount of comments that describe the code's purpose, rather than how it works, is needed.  There are are additional rules on syntax, capitalization and indenting that form some of the standards of ROC.

Fragile code is the opposite of robust code. Matt Bishop of UC Davis defined robust code, or bomb-proof programming as, "a style of programming that prevents abnormal termination or unexpected actions. Basically, it requires code to handle bad (invalid or absurd) inputs in a reasonable way. If an internal error occurs, the program or library terminates gracefully, and provides enough information so the programmer can debug the program or routine." When a developer has to make changes to a fragile code base there is a higher probability that they could, in turn, make the code more fragile.

How can we as Manager or Developers help eliminate or reduce the impact from these challenges? There aren't any easy answers, but there are some potential opportunities to make improvement. Unrealistic expectations and requirement changes are not going way. They are part of the territory of development, and solutions are better if they're approached through the acceptance of factors which are out of our control. Agile Development presents an opportunity to deal with such an environment.

Around 2001, a group of developers got together and created the Agile Manifesto:

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

And from this, a set of 12 principles:

  1. Customer satisfaction by early and continuous delivery of valuable software
  2. Welcome changing requirements, even in late development
  3. Working software is delivered frequently (weeks rather than months)
  4. Close, daily cooperation between business people and developers
  5. Projects are built around motivated individuals, who should be trusted
  6. Face-to-face conversation is the best form of communication (co-location)
  7. Working software is the principal measure of progress
  8. Sustainable development, able to maintain a constant pace
  9. Continuous attention to technical excellence and good design
  10. Simplicity—the art of maximizing the amount of work not done—is essential
  11. Best architectures, requirements, and designs emerge from self-organizing teams
  12. Regularly, the team reflects on how to become more effective, and adjusts accordingly

Agile proposes an iterative process through releases (rather than big bang projects), with close collaboration between the business and developers that expects and welcomes changes. Smaller release with multiple iterations reduces the complexity and potential pitfalls from changes using the big bang and waterfall methodologies. Agile tends to minimize administration and non-value added by replacing those with increased collaboration and trust, relying on the self-organizing and self-improvement that good teams seem to bring about.


The Agile methodology has been growing in popularity as a replacement to the waterfall methodology at many companies, which is reflected in this pie chart from a recent survey conducted by HP.

Agile doesn't necessarily remove the six biggest challenges developers stated they face, but it can provide a process that better handles the real world hurdles of any development endeavor. Although many companies are leaning towards or pursuing hybrids that include Agile, there is probably a learning curve making it as effective as it may be for the early adopters. Of course, something that lies ahead may require an improved Agile, or maybe even something all together different, but any process that can effectively manage change has some staying power in the worlds of business and technology.