Giving back - programming is a community effort

09.12.2019
martin friedel
coding

We are a software company, creating software that uses a set of technologies to solve the needs of our customers. But in fact there is more to this story, especially concerning developing web applications in the browser.

Our tech stack consists primarily of React for the frontend (what goes on in the browser) and Go for the backend (what happens on our servers). Go is a general purpose language that can do a lot of things. React is a technology to allow construction of very complex software running in a browser. It compiles down to the mix of HTML/JS/CSS that the browsers know internally - but that is not very well suited to building larger applications with directly.

The Go universe is simple

Now Go is about as minimalistic as you can get and we love that. You install Go and it comes with everything you need - compiler, package management, test framework, code formatter and more - all built in. The standard library that Go brings along already contains all essential features and you usually end up only installing a very few and specific libraries i.e. to connect to the specific database you are using.

For a simple task you can truly have a project that consists of one source file. Go compiles this one file to one executable that is the finished working software you can run.

The JavaScript universe is complex

For the frontend it is a different story. The base technologies are originally made for displaying content - HTML stands for "Hypertext Markup Language" - and were designed for websites containing text, headlines, images and links. The browsers have since then added so many powerful additions to their arsenal, that today you can use a browser to build virtually anything from content to application interfaces to 3D games. But this legacy provides some real challenges too. JavaScript was originally intended to handle very simple tasks inside a website and not large scale software projects, but that is the direction that web application development is taking - more frontend, less backend.

So where Go delivers a complete package with all the tools right out of the box, JavaScript is a different story. The browsers supply the bare essentials - the language working inside the browser. Then millions of developers create building blocks on top of that base foundation, which is what we use today to build web applications. (The developer tools in modern browsers are truly amazing too, but they help you with running, analyzing or debugging your code, not with building it).

npm packages

So for JavaScript this has spawned the largest collection of building blocks (so called "packages", which are programming libraries) of any programming language ecosystem ever. The central nodeJS package repository contains the staggering number of over one million such packages now (admittedly, that million also contains such useful things as the hodor package).

So frontend development today really is its own kind of adventure. React itself has about 80 dependencies (packages that it is based on and that need to be installed to use it) but those packages themselves again have dependencies of their own. If you create a fresh empty React application you will download over 1000 packages as dependencies, which is more than 200 megabytes in 30.000 files. Of course, most of these are packages used during development and not included in the final result that runs in the browser in the end. But with such a big dependency tree, there is a lot of complexity in the basic foundation itself (it gets worse by the way - two packages can depend on the same other package - but with different versions! How much fun is that? Well, thankfully there are tools, that have found ways to handle that gracefully, but under the hood this kind of thing is present and being handled).

So that is the ecosystem we are working in to produce web applications with. Its both humbling to see the sheer amount of effort that went into these packages that were created to help build almost anything and enable this kind of programming. But we also experience regularly that this complexity brings real pitfalls into our day to day work, where some update to one package breaks something else.

Change is rapid

And on top of that, new libraries, new ideas inside of libraries, new best practices are emerging constantly. The whole ecosystem is ever changing. React has a very good track record of never introducing breaking changes that would force you to rewrite an existing project. But there have been updates with great new features that make you want to rewrite your code. And thousands of libraries that use React as a foundation have gone ahead and rewritten their code to fit these new possiblities. And sometimes that means some changes for you too.

You can freeze the versions of all your packages to have a stable envirnoment for how long you want, but at some point you will want to upgrade to keep up with bug or security fixes.

Conquer this complexity by giving back

And this brings me to one of the reasons for this blog to exist. We truly could not build the software for our customers if there wasn't millions of other developers out there, wrestling with the same questions or discovering similar problems, figuring out ways to fix it and sharing those discoveries. Countless hours are spent on such problems, but by sharing the knowledge someone has gained, we as a community of developers are crowdfixing these problems to keep the complexity in check.

Especially in front end development, we rely on our fellow programmers. This blog is our vehicle for giving back: for sharing our experiences, bug fixes, discoveries, for explaining some of our reasonings, some of our findings.

If you read one of our posts and it saves you a few hours of figuring the stuff out yourself, then consider that our thank you to all the hours we have saved so far, because someone shared their knowledge.