Author Archives: paulborile

About paulborile

I’m a multi-skilled IT professional with a good all-round supervisory and technical expertise. Extensive, 20+ years of professional experience in software development allowed me to investigate computer science and software engineering inside out. During these years I built up a solid base of design patterns, software architectures and programming languages such as C/C++, Golang, Java, Python, SQL, Assembly (and many others). I worked on mission-critical and multi-channel applications, applying distributed computing, messaging, image/data processing and computer graphics techniques. I faced both architecture design and systems rearchitecting, microservices introduction and technology migration as well as company wide adoption of new technologies/methodologies multiple times. As an entrepreneur I have built and grown teams and development organizations from the ground up (internal/out sourced/at customer site) focusing on software engineering methodologies as well as recruiting, budget/financial control and operations support. I am particularly interested in software testing methodologies, software quality metrics and tools to make software development faster and better. Currently leading the Italian development team for ScientiaMobile Inc, a Reston (US) based startup focused on image optimizing CDN and mobile detection technologies and services. Born in Dearborn Michigan and living in Italy since many years now I speak fluently both English and Italian, studied French and learned some Russian while working for some time for a Olivetti/Aeroflot project.

wtfperminute

What’s wrong with word “simple” ? ( on over engineering code )

Over engineering code is a common plague that leads to a number of unwanted side effects in your software ranch. Consider it just a form of bad software design.

This post does not want to be an exhaustive guide on over engineering but with the help of many quotes  from different sources, I’ll try to summarize the impacts, when you can have evidence of over engineered code and the causes (to try to avoid them).

Impacts

  • unneccessary complexity => less agility in software
  • more code, more bugs
  • more code, more unit tests, more development time
  • performance impacts
  • longer catch up time for newcomers
  • difficult maintenance, fixing bugs requires more time than it should
  • small changes require big efforts and are prone to having many bugs
  • unwanted features are present even though there is not reference in requirements/backlogs and so you have to maintain them even if they are not used
  • false perception of complexity in the code that leads to over inflated estimates for fixes, changes.
  • extending an over engineered project makes extensions more complex than necessary
  • dismantling of over engineered code takes way longer than it took to over engineer it
  • Need for complete rewrite over time to reduce unneeded complexity when maintenance times become higher than rewrite time This is an interesting topic itself and needs a separate post)

Concrete symptons  of over engineered code

Really difficult here to point out some easy and simple guidelines to recognize over engineered code. Often it is an individual perception and this does not help. I’ll try :

  • When a well experienced coder, which has been working on that piece of code for at least an year, still takes way more than necessary/affordable to figure out where/how a specific feature works. I’m trying to find a better explanation for this concept
  • In code : using a factory even if it is making type of objects
  • In code : use an interface even if it is actually going to be implemented by just one class
  • From PrematureGeneralization

“This is really going to be a clean framework. I’ll make an abstract class out of this part so that folks can subclass it later, and I’ll put in a bunch of well-commented overridable hooks in the concrete subclasses so that folks can use them as templates, and just in case somebody ever needs to build special debug subclasses, I’ll put in extra stubs over there (somebody will thank me for ’em one of these days). Yeah, this is really going to be neat.Thus is bloated software produced when our artistic sense gets the better of us. — DaveSmith “

Some other signs of over engineering taken from stackoverflow :

One very strong warning sign of over engineering is when everything goes through so much indirection that it’s hard to find the piece of code that actually implements some concrete, domain-level piece of functionality. If you find that most of your functions do very little concrete work and just call other virtual functions, you may have a problem.

https://softwareengineering.stackexchange.com/questions/193929/too-object-oriented?newreg=f7722bef1ba7496e9d8b4e7d328cadf1

[..] Make factories where the factories make more then one type of object. Use dependency injection, where it immediately shows benefits. Make interfaces that are actually going to be implemented by more then one class [..] What I see too often in “true OO” is that advanced techniques are used to solve really simple problems in an overly complex way [..]

Causes of over engineering :

Accidental vs essential complexity

Interesting article, some really good points :

Software should behave predictably and accomplish its goals without too many surprises (that is, outages in production). The number of surprises directly correlates with the amount of unnecessary complexity found in a project. It’s therefore crucial to think about accidental complexity and essential complexity:

Accidental complexity relates to problems which engineers create and can fix, [whereas] essential complexity is caused by the problem to be solved, and nothing can remove it
— Fred Brooks in his seminal “No Silver Bullet” essay

Another interesting point on when a coder start to write unnecessary code :

Boredom is good precursor to over-engineered code. I’ll admit, when I got my first job, I felt so underutilized. I was just bored. And when I got bored, I wrote code. Not just any code — CATHEDRALS OF CODE.

No seriously, I had a mental picture of my code and abstractions as large towers with golden jutting spires, flying buttresses of glassy onyx, a wonderful vault supporting by arched domes topped with beautiful geometrical tracery, etc etc etc.

It was really fascinating to see the patterns working together for myself, but in retrospect, I am completely ashamed of the ungodly mess I left behind.

If you’re writing your own frameworks and DSLs code to while away the less stimulating hours at work, just stop. Time is better spent reading Wards Wiki, or writing an open source book, or you may just want to ask management for more work.

Some theory on why/when over engineering happens

  • Second system effect : you wrote a first good, simple, performant version of a product and it is getting traction so you decide to write 2.0 version of it which does everything.
  • feature creep : you just keep adding features that are used by less than 1 % of the user base/use cases.
  • inner-platform effect : you are making your system so customizable to become a replica of the features provided by the sw component of dev environment you are using.

Notes on over engineering : https://blog.matt-rickard.com/p/stop-overengineering

Avoid unnecessary abstractions (and some other usefull info) : Techniques to avoid over engineering : https://betterprogramming.pub/3-tips-to-adopt-a-simplicity-mindset-when-designing-software-711b95328062

Beware of complicated solutions (that someone was paid to find) @ntaleb

I’m still working on this point : here are some  references to interesting concepts

The KISS principle

YAGNI : Your Arent Gonna Need It

DRY : Don’t repeat yourself

Simplicity is a prerequisite for reliability

The “Fix it or rewrite it from scratch” dilemma

I’m sure you found yourself in the situation of having to fix a piece of code that :

  • was not written by you
  • has not a clear set of requirements or the requirements are clear but the code is unnecessarily complex (at a first glance looks like you encountered the dreaded “big ball of mud”)
  • Nobody knows exactly the requirements anymore so you’ll have to reverse engineer from the code the behavior
  • Tests are missing or trivial and don’t help in understanding what that piece of code should do

My first question here is to understand where is the break even between fix and rewrite. I tend to prefer rewriting code if :

  • besides needing fixes there are also performance problems
  • code needs to be fixed/changed often
  • code is a company asset
  • code complexity is exposed to customers
  • coders turnover is high

Remember that :

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
— Kernighan and Plauger in “The Elements of Programming Style”

Let’s try to add some information. Pros and Cons for rewriting instead of fixing :

Cons :

  • Time/cost : rewriting takes time, probably more than a “quick and dirty fix”. The real cost analysis here is “how much faster would we make fixes if we had a clean code base?”. How many fixes we perform every year ?

Pros :

  • Coders will not have to work on a “big ball of mud” : this causes frustration and in the end lowers productivity (Time/Cost–)
  • New code base will probably be smaller (requirements sediments will get cleaned up), so less tests need to be maintained (Time/Cost –)
  • New code base will probably be faster (Cost–)
  • Newcomers will have short catch up time on the new codebase ( old codebase Time/Cost –)

So in the end all pros of rewriting lead to less time for maintenance/enhancements and less costs. In the end “Quality is Free” ( https://www.stellman-greene.com/2006/03/29/quality-really-is-free/ ) according to these guys.

 

SmartOS is like having vagrant/docker/virtualbox builtin in your OS

I recently had a chance to try out SmartOS after my last experience in application porting on it (2012) and I was impressed by the virtualization features that are made available directly by the os. One set of commands allows downloading images, running either vm like container or namespace like  containers. Zones also allow you to run debian/centos applications on SmartOS inside a lx zone with system call translation … Awesome.

If your interested in digging deeper take a look at this post from Tim Boudreau.

 

Yes, I like go programming language

Yes, I like Go programming language. I’m liking it so much that I have to resist from being a fan boy. I’m trying to understand where all this enthusiasm is coming from (I’m a seasoned coder)  so here’s an attempt to find why :

  1. Code readability — and maintainability — first, language features second
  2. Integrated test environment : go test <package> executes all tests for the package. Unit testing features are builtin.
  3. Code Coverage is builtin (with some limitations, for example if you use cgo it will not work).
  4. Integrated tool chain : no need to have makefile at the cost of rigid hierarchy of data.
  5. Exhaustive standard library containing everything you need to do server-side/network programming
  6. Good Multi-threading features/model included in language (sync package, goroutines, channels), fast goroutines thanks to segmented stack implementation.
  7. Basic set of OOP features, centered on composition, not inheritance : you won’t be able to mess up your code at the cost of not being perceived as an OO language by OO fanboys. For more details on whether go is oo or not go here.
  8. Go is backed by some Famous Names in computing, and this inspires confidence.
  9. CamelCase 🙂 ? Naaah, I hate camel case but I like the choice of having standard style, comments, indent; all supported by the language via go fmt package so that all code will look coherent.

So basically I like the fact that Go is a very opinionated language. You might like the single decisions or not but what I like most is that someone took care of taking them for you (so you don’t have to enforce them team wide or company wide)
Interesting read also on how and why go was born : quoting from Rob Pike speech at go conference SF 2012 :

“To put it another way, oversimplifying of course:

Python and Ruby programmers come to Go because they don’t have to surrender much expressiveness, but gain performance and get to play with concurrency.

C++ programmers don’t come to Go because they have fought hard to gain exquisite control of their programming domain, and don’t want to surrender any of it. To them, software isn’t just about getting the job done, it’s about doing it a certain way.

The issue, then, is that Go’s success would contradict their world view.

And we should have realized that from the beginning. People who are excited about C++11’s new features are not going to care about a language that has so much less.  Even if, in the end, it offers so much more.”

Top software coders bad mental habits :-)

New year is time for self-examination; one of the most frustrating things for a coder is writing code with bugs and bugs are almost always directly related to bad mental habits imho. This is not a complete list of any kind, it is a set of well known and widespread ones that you for sure have already encountered in your coder life. I’m writing them down just as a reminder for myself :

  1. The “Let’s do it this way for now (I know this cannot be the final way of doing it), because I don’t want to stop 5 minutes and think about it” attitude. This is the worst of all bad habits in my opinion. It is this attitude that generates the most of production bugs because the only moment you had to focus on that specific issue you decided to skip over it, for the sake of continuity in your mental path (which is a good thing by itself but bad if not derogable ). That moment will never come again, that “preliminary” code will go strait to production and that issue will never ever be taken into account again until it generates a bug.
  2. Using the “quick and dirty” way to do things even when there is no real need for that type of approach. This is related to the fact that we are almost always pushed to deliver fast and after time the “quick and dirty” approach becomes the standard one, always, regardless of requirements.
  3. Unreadable code : this does not necessarily generate bugs but makes it difficult to fix them.It is caused by :
    1. coder EGO : “nobody will ever be able to understand my code unless spending an hour over 10 lines”. I will be the only able to maintain it.
    2. “This way is faster, (probably) ” (note probably, because nobody is ever measuring code speed). Modern compilers/cpus do things that we can’t imagine in terms of optimization, but “I can do better”.
  4. Comment out unused code, or worse, gate it with a feature flag. Code that has no purpose is a major source of distraction and confusion. Today’s version control systems make it easy to revert any changes; there’s no reason not to remove dead code and other bloat.
  5. Over engineering code or overdoing features : this one is so big that needs a separate post to handle it but we might try to summarize it with
    1. more code, more bugs
    2. more code, more tests to make, more time
  6. “I’m gonna do this in 10 minutes” attitude