Author Archives: paulborile

About paulborile

I’m a multi-skilled IT professional with a good all-round supervisory and technical expertise. Extensive, 20+ years of professional experience in software development allowed me to investigate computer science and software engineering inside out. During these years I built up a solid base of design patterns, software architectures and programming languages such as C/C++, Golang, Java, Python, SQL, Assembly (and many others). I worked on mission-critical and multi-channel applications, applying distributed computing, messaging, image/data processing and computer graphics techniques. I faced both architecture design and systems rearchitecting, microservices introduction and technology migration as well as company wide adoption of new technologies/methodologies multiple times. As an entrepreneur I have built and grown teams and development organizations from the ground up (internal/out sourced/at customer site) focusing on software engineering methodologies as well as recruiting, budget/financial control and operations support. I am particularly interested in software testing methodologies, software quality metrics and tools to make software development faster and better. Currently leading the Italian development team for ScientiaMobile Inc, a Reston (US) based startup focused on image optimizing CDN and mobile detection technologies and services. Born in Dearborn Michigan and living in Italy since many years now I speak fluently both English and Italian, studied French and learned some Russian while working for some time for a Olivetti/Aeroflot project.

grigna

On the importance of design in software

Many time we face situation were productivity of a software team is impaired by initial flaws in system design. These initial flaws require too much time to be completely removed (the effort of complete rewrite is only marginally touched here and is normally and indication of other problems in your software project )

So, in order for a software team to be able to work at its maximum, good design is a must. Design is much more responsible for productivity than any single coder, scrum master, product manager or development methodology/language in the sense that bad design can take an all-star team using top tools/methodology to perform badly.

There were times when it was taken for granted that before building a system, it was necessary to design it. That time looks gone (even though Agile does not explicitly prohibit making good design). The general idea seems to be that there is “never enough time to do something right, but there’s always enough time to do it over” again.

But what is design in software ?

That kind of intellectual activity which creates a whole from its diverse parts may be called the design of a system

Melvin E. Conway, How Do Committees Invent?

So you compose parts to make the whole system. How you do this ?

It is basically decomposition to generate single parts that acting together will generate the goal. The 2 tools you will need as a designer are :

  • decomposition
  • composition

 But first of all you need :

  • understanding of the system boundaries : the borders of the nation your system lives in
  • understanding the global scenario of what your system is going to do. Designing subsystems without knowing the whole picture is not a good idea.

Why decomposition of a system into smaller components (services) is good for you :

  • information hiding : well identified interfaces create a contract that the component has to provide, hiding any network/system implementation details. This will make the component implementation completely free to be modified/enhanced as long as the new implementation is compliant with the contract.
  • less development time : separate components/services can be developed in parallel since they don’t require (or require little) external dependencies, so less contention between engineers. Every component is independent on integration and performance test that can be developed autonomously.
  • scalability : with little effort in design every component might be deployed in a way to be one of many instances that will make that service scalable when traffic increases.
  • clarity : the system could be studied a component (service) at a time with the result that the whole system could be better designed because it was better understood

We could go on here and analyze what are the criteria to be used in decomposing a system into services. I’ll leave this to another moment and you can find some interesting notes here :

https://blog.acolyer.org/2016/09/05/on-the-criteria-to-be-used-in-decomposing-systems-into-modules/

What I would like to stress is that design is a fundamental phase in software engineering. You can’t just skip it and pretend that since you have a good team you’ll get a good job done.

Thanks to the following for inspiring me on this post :

https://en.wikipedia.org/wiki/Melvin_Conway

https://blog.acolyer.org/2019/12/13/how-do-committees-invent/

https://blog.acolyer.org/2016/09/05/on-the-criteria-to-be-used-in-decomposing-systems-into-modules/

Benchmarking golang code

Let’s say that you want to know if EncodeToString is faster than fmt.Sprintf : you will need to compare the speed of this method 

func Md5Encode(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := hex.EncodeToString(md5HashInBytes[:])
	return md5HashInString
}

with this other one 

func Md5EncodeFmt(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := fmt.Sprintf("%x", md5HashInBytes)
	return md5HashInString
}

Go provides benchmarking features in the testing package which is pretty usefull :

func BenchmarkMd5EncodeFmt(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5EncodeFmt("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

func BenchmarkMd5Encode(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5Encode("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

Run 

$ go test -bench=.
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1894791	       625 ns/op
BenchmarkMd5Encode-8      	 3068509	       363 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	3.342s

Run 3 times the benchmarks :

$ go test -count 3 -bench=. 
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1882105	       627 ns/op
BenchmarkMd5EncodeFmt-8   	 1918942	       624 ns/op
BenchmarkMd5EncodeFmt-8   	 1902894	       625 ns/op
BenchmarkMd5Encode-8      	 3139585	       386 ns/op
BenchmarkMd5Encode-8      	 2937154	       397 ns/op
BenchmarkMd5Encode-8      	 3009801	       380 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	10.217s

EncodeToString() makes your method almost twice faster !

In the case you need to isolate the code that will be measured from some init/fixture code you can use ResetTimer() and StopTimer() to accurately isolate what you want to measure : 

func BenchmarkMd5EncodeFmt(b *testing.B) {
	// Some init code
	initRandomBenchStrings()
	b.ResetTimer()
	for n := 0; n < b.N; n++ {
		Md5EncodeFmt(getRandomstrings())
	}
	b.StopTimer()
	// some final code
}

If the function you’re measuring is very slow you might want to increase to maximum time of excecution of the benchmark (default is 1 s) with -benchtime=20s

Recap

  • without any benchmarks : go test .
  • with benchmarks (time) : go test -bench .
  • with benchmarks (time and memory) : go test -bench . -benchmem

The argument following -bench is a regular expression. All benchmark functions whose names match are executed. The . in the previous examples isn’t the current directory but a pattern matching all tests. To run a specific benchmark, use the regexp : -bench Suite (means everything containing Suite).

(thanks to https://github.com/samonzeweb/profilinggo )

Thanks year 2000 : less is (immensely) more (the 90s produced a lot of crap)

Thanks to god after year 2000 information technology has started moving towards more pragmatic, simple and effective tools and languages. Some examples that in my opinion make this evident : 

Languages and language tools

  • go, rust, swift are all born with the goal of simplifying their direct parents (c++, objectiveC) and removing their pitfalls.
  • UML abandoned : this is a relief for all coders which had to deal with it. I don’t know anyone using it nowadays.
  • git : finally some one (thanks Linux Torvalds) simplified svn/sourcesafe by putting features that are needed by developers in a clear, pretty intuitive command line interface
  • atom/sublime : reaction to the complexity of Visual Studio, IBM Rational, Eclipse ? I think yes

Databases

  • Key-value stores/noSQL are just taking ER/SQL model and making it simpler, providing only the features needed in 99% of the applications. Boyce-Codd normal form is pretty nice and interesting but in real world applications you’ll never use it. 
  • Object Databases completely disappeared and in some way also the idea that OO methodology/hierarchy could be applied everywhere (just because you are where using OO languages)

Virtualization

  • docker/rkt are slim alternatives to virtualization and virtual machines

Architectures

  • plain old REST API aren’t just a simple way for doing things without having to Corba/Soap ?
  • gRPC : provides corba like features while being 1 order of magnitude more efficent and portable on any platform.

What I’m saying is that the 90s produced a lot of unnecessarily complicated tools and technology which developer just did not need/like which is being progressively substituted with simpler stuff.

Interesting to note that the phrase “Less is more” is originally attributed to Mies Van Der Rohe for his minimalism in architecture design . Looks at his buildings : nothing more than necessary and functional elements are present.

But before him Leonardo da Vinci : “Simplicity is the Ultimate Sophistication”

Antoine de Saint-Exupéry – “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.”

11-36 Cassette on Tiagra RD 4700 (hacking a tiagra rear derailleur)

Tiagra rear derailleur is often mounted on gravel bikes but I did not like its limit to 34 maximum cog size. My hackers attitude comes out in these situations (and I don’t care breaking the component warranty) so I decided to modify the derailleur to hold larger cogs.

The problem is that the B-Screw is short and won’t allow you to increase the distanze between the lower cogs from the upper cassette. While some tutorial suggest to put a longer b-screw, I tried what can be seen below :

I used a chain pin and insert it to make the mechanical stop longer. Some thread blocker or resin to keep it still and you’re set.

It works 🙂

Milan skies after fires in the alps, 2018

Decarbonize!

Friday for Future is running and I feel the need of making sure (firstly to myself) that the process that will bring us totally away from fossil fuel consumption is possible, maybe long, but possible.

Decarbonization (this is the name given to the biggest revamping project in the world) is possible; will require money and time; will require the mutual work of Politics, Science and Industry toward the goals of :

  • producing electricity totally from renewable sources, decentralize prodution
  • reducing the energy consumption in all areas were this is possible
  • decentralize smart grid and electricity storage development
  • substitute direct fossil fuel consumption with renewable alternatives
  • stop deforestation process
  • substitute fossil fuel derived products with fossil derived recycled ones (or carbon free ones if possible)
  • 100 % recycle, waste to energy for the non recyclable

Ambitious plan ? I think this is the biggest revamping project you can immagine and it is already running but I think that the message we all sent last last friday is that we need to ‘deliver’ sooner 🙂

Producing electricity totaly from renewable sources

48 % of CO2 is emitted producing heat or electricity. Many countries are already active in the area of producing electricity from renewables, take Germany for example. 7 year ago (just after fukushima) Germany started phase out of nuclear power by incrementing the share of energy produced by renewables. Some data :

source

In 1 year Germany increased production from wind energ for example by 20 GWh. Continue this for 10 years and your reach more than half the whole country energy requirements. In fact Germany has also started a plan for removing coal in energy production.

In the first 6 months of 2019 Germany has produced more energy from renewables than from fossile/nuclear : here for some references.

Energy efficency

Again from Germany, a national plan to increase the efficency of systems in all areas which is estimating to produce a saving of 12 to 20% over 2020. Reducing the current energy footprint is fundamental for allowing new segments of activities to start using clean energy (think at electrical traction in automotive which is going to increase national demand)

Decentralize smart grid and electricity storage development

The example here comes form Australia were private energy company GreenSync is stimulating customers to setup local electricity storage to be used when there is shortage of power on the grid. Customers are being paid for the storage. For reason not known to me the biggest development in decentralized grid and storage is taking place in Australia and Japan.

Substitute direct fossil fuel consumption with renewable alternatives, limit impatc of direct CO2 emission

This is probably the biggest task in the project because it is spread over a tens of different segments which need to be revamped to achieve the goal :

Road Transportation : around 15% of total CO2 emissions. Redesigning this segment is going to be one of most serious tasks : cars and trucks make up 1/3 of the co2 emissions in countries like US and it is mostly a consumer segment. Battery powered electric cars, pickups and trucks seems to be the directions with Tesla, the real game changer, paving the road. All automotive industry is trying to catchup. 44 Billion investments announces by Volkswagen group over the next 5 years.

Agricolture : How much CO2 is produced by agricolture is the most controversial issue with estimates ranging from 13% of total CO2 emissions to 18% on fao docs, up to 51% including the effect of not having forests where we make food for cows, pigs and chicken. These comes mainly from Cattle belching (CH4) and the addition of natural or synthetic fertilizers and wastes to soils. Here the only possible change is reducing the use of fertilizers and reduce cattle breeding by eating less meat. Read Jonathan Safran Foer book if you want to dig into this more.

Maritime Transportation : 5% of total CO2 emissions, The world’s merchant fleet consists of around 100,000 ships and these are estimated to consume 250 million tonnes of bunker fuel annually. Just one Capesize Bulk Carrier or Bulker can use 40 metric tonnes or fuel or more a day leading  to an annual fuel consumption of approximately 10,400 tonnes. This results in the emission of around 32,988 tonnes of CO2 and 959 tonnes of SOx or more. This is just from one ship. Still no real prototypes afaik in this area but good project and potential around with project like Acquarius.

Air Transportation : 2% to 3.5% of total CO2 emissions . Various activities undergoing reduction of carbon footprint in aviation.

Substitute fossil fuel derived products with fossil derived recycled ones (or carbon free ones if possible)

This is probably the biggest task in the project because it is spread over a tens of different segments which need to be revamped to achieve the goal :

  • Plastics
  • Lubricants
  • Process Chemicals
  • Carpeting
  • Pharmaceuticals
  • Rubber Goods
  • Adhesives
  • Cosmetics
  • Footwear
  • Paints
  • Detergents
  • Inks
  • Sealants
  • Fragrances
  • Solvents
  • Caulking
  • Compounds
  • Fertilizers
  • Fibers
  • Tires

This point will require a complete structured analysis by its own. International energy agency dedicates a complete section on petrochemicals. They are not easy to replace : recycle will be the solution while Science and Industry find better substitutes.

I’ll stop here at least for now : the message I’m trying to share is that the matter is highly complex and cannot be simplified by just switching off air conditioning or doing these kind of things.

ALL activities have to be done at the same time (thanks Greta for having said this) and Politics IS the driver for all of them.