A Software Engineer trying to make things better

I've recently noticed a fun thing happening in technical circles. Especially among professionals part of it, where guys to what business what them to do. I call it “labeling”.

As I see, people identify themselves with technology they use at work. Like, “I am a Ruby Developer”, or “I am a Java Architect” and so on. And people, who tell such things, tend to put labels other people as well. It sounds like if some person works with one part of tech, he can't learn other parts. Backend Web developer can't get working with Frontend Web, or FullStack Node Developer can't learn python.

It frustrates sometimes, because is very stupid. It's like a cook that chooses to use knife and only knife. He can achieve truly perfection with the tool, but he may still cook röt dishes, because he can't stove, or choose spices correctly. Similarly, every single programming language and utility set is just part of toolbox of good developer or engineer.

Moreover, even if some language didn't fit into your toolbox – you can learn from any language you learn a lot. Half a year ago I've learnt Scala programming language, and learnt a lot from it. I've learnt, how strong enough typing system can make life way easier and simpler, what is FRM and more. Although it seems that it will not stick in my toolbox for now, but this experience showed me about such thing as Abstract Algebra and Category Theory. Otherwise I could learn about it way later or even never.

Please, stop labeling yourself to the tool. You deserve better just to be a servant of programming language.

If you have some thoughts on subject, feel free to contact me any suitable way

The thing in software world that is unlike to others, is that you can achieve perfection at some point. You can build perfect system, that will work fast, low on memory and easy to use and extend by user. At least we can think so and proof using maths, which is the source of this feeling in IT. Generally, all the programs are models of some processes we tend to see in world surrounding us. Task managers, calendar organizers are models of organization systems. Messengers and social media are models of people communicating. There are more of those models of real life entities.

But, in fact, all the models have their issues. Mostly they are some projections that shows only some properties of process it models. So, for sure, you can't achieve perfection, because the closer you are, the less of image is left and more of real world you get. It can take lots of time and effort to implement all the features people will ever need.

“Why you are telling me this?”, you probably will ask. Well, the thing I am talking about now is: “Perfection is hard to achieve, and mostly is not required at all”.

I've switched to Windows for two weeks lately. I've been using it for different sorts of things: gaming, general web browsing, software engineering and performing some ops tasks. And after two weeks, I've switched back to Linux even though while installing Windows I made a promise to myself that I will focus on getting things done instead of making things work perfectly fast and suit me. And I did this because I wanted to keep the promise.

Windows became pretty nice platform nowadays. They offer pretty decent and performant workstation that fill most needs in computer:

  • web browser works fine (however I've switched to firefox, because it is in fact more lightweight than chrome or edge)
  • multiple workspaces/desktops work very good for my workflow
  • there is plenty of quality development tools that don’t suffer in usability out of difference of OS
  • gaming is still the best and even with recent improvements in face of Proton, things are not that great

But there is one thing that makes this all irrelevant for me. I do software development for web, and *nix-compillant operating systems are first-class citizens there. So, the windows system has gotten some duality of things happening around: some things worked withing windows environment and some – withing linux environment. And lots of time, I was just trying to find a good windows-native way to deal with things. For instance, I couldn't find a good way to work with ssh keys for windows that could be used by git, vscode and intellij at the same time (without using 3rd party tool PuTTY). Mostly one of things didn't work at all. And bash scripts written for linux didn't work exactly like they did in linux. And the containers in fact were running in virtual machine... which made them not that lightweight. I know about WSL, but it is like wine today: generally working, but not exactly. And WSL2 is still a virtual machine. I did pretty same thing with VirtualBox and it worked way better than official Docker for Windows.

So, Linux is in fact the way of getting things done for me. I still can do gaming, general browsing and software engineering. Yes, gaming is not that awesome on Linux, but good enough for me. For all the other tasks, I achieve more by doing in fact less.

Making conclusion, you have to understand better, which things give you most of issues and which give you more possibilities. The “easier” and more “user friendly” system may be worse choice for you if it will make your work done slower or worse quality. And there is no shame at all in using proprietary systems or even those who track you if it is really the best way of getting things done. See software selection as an investment. If you make a bad one, you may end up spending lots of time doing things that are not actually a problem for anybody.

Lately I read a post at mastodon, where guy told, that his life is in browser: he manages project there, does code review, read articles and news, even writes code in browser-like environment. It's kinda great, because there can be developed greater products cross-platform out of box, fast and beautiful. And people spend less time adopting one single piece of software for specific platform.

I totally like it.

Yet still prefering desktop applications. And the reason is very simple: it is better. Really native (I mean really native apps, like built using native tools) applications can work faster, use resources more effective and do more things thanks to deeper integration with environment it's running on.

“Duh, JavaScript is really fast nowadays”, you can say. And it's truth. Yet, it is a wrapper over wrapper. It's dynamic nature makes bad joke of it, killing it's performance in bad hands. I've written some heavy things computations in JS which were running slower than in Python. And, moreover, usage of ES2018+ features make it run even more slow... And the memory – the simplest application written using Electron uses at least 30-40MB of ram. While simple native app can use about 1-2MB depending on which technology you use.

About beauty of whole system. It is easier to make beautiful web application using web technologies. But, mostly it doesn't fit the system's style at all. Same issue with Qt+QML applications. It just look like something taken from out of ecosystem and make all the things look strange. Also, web technologies have huge issues with building layout (yeah, I know about flexbox and grids, but they are pretty new here): they are sometimes ineffective and not so responsive as people do it with hacks. Qt QML has much better one, yet it has some issues, but it's not about them.

Integration of system does not end on beauty. It's also effective resource management, minimizing actions without purpose. For instance, I used to make awesome pkill -INT slack to increase battery lifetime about 2x while it is not active. Just because it keeps comsuming CPU just while it is open. I would kill it if startup time was not so awfully slow. Then comes configuration handling, database storage and more. It also can be reusage of libraries people have already installed inside the system, which will make application smaller, faster and do more.

Yet, web technologies allowed to make a lot of things, inavailable before. Web became and operating system, shared to any platform and user. A computer without browser literally becomes 10x less useful than without it. It's like Java's manthra “Write once, run everywhere”, but for real, with more beauty, less pain and easier. Way easier.

But when it comes to advanced usage, native apps should be preferred to web ones, because it will allow you to consume your computing power and resources more effectively. Just try and you will see, how huge amount of computer resources can be saved just for running a native version of application instead of web one.

Self-hosting becomes more and more widely used by non-tech-industry people. They self-host rss readers, site saving systems, social ActivityPub/LitePub compatible applications and more. Thanks to wide usage of Docker and other containerization systems it became much easier to manage selfhosted items than it was 10 years ago.

Yet, sometimes it is still hard to do, especially when people want to run an application servers on bare metal, without containers or any kind of virtualization. And sometimes, it is painful to do it, even for software engineers, who developed it.

Why? Very simple – most times it takes much steps to perform to set it up.

You see, mostly you have to set up not just self-hosted service instance, but also databases like PostgreSQL or MySQL, Redis, ElasticSearch etc; write some kind of configuration file for an app; if service has microservice architecture, you have to set up every single microservice.

Of course, you can use Docker, Ubuntu Snap or other technologies. Yet, if you use them, it will take much more space, it will take a bit more time to start a container (duh, you have to download it, unpack or build from scratch). It is inappropriate for some people, for example, I run every thing I host on bare metal VPS I bought from provider.

Let's get back to how to make a service be easier to self-host: what you can do as a developer make it happen.

1 – Make your application be shipped as one executable file. Mostly it applies to scriptable languages like python, ruby, javascript and others.

It just grinds my gears when I see such thing with django packages, like this:

Now run python migrate

It's stupid – to put all source code in files so people have to install all the dependencies by themselves. Why not to package an application in single executable file that already has all the dependencies inside? It would make people 100500 times happier, because not all self-hosters are developers to read and change the code.

Also, there are some external bundlers that hide all amount of this things under the hood. It's nice, yet it is still shitty thing. Why in Equestria do you need an external bundler application to run an application? I don't want to install another package manager for javascript to run an application. I just want run an application, and that's all.

2 – External services dependencies, like databases or else. It is a painful one for developers. Yet, the best way to make service easy hostable, you probably should allow support of most popular alternative databases. Or make dependency on it optional.

Why it is painful? Because it will make code more complex. It will require more tests connected to environmental things. Why it should be done? Because it will allow people to use tools they know better, or decrease amount of additional services to support.

For instance, let's assume that you want to set up RSS feed reader. One, that fits the best for your need has a requirement of elasticsearch, and it is used only to index and search entries. Then, why do you need elasticsearch if you are not going to search for something at all? You just don't set it up, and service doesn't provide search at all.

Or, you already have running PostgreSQL database, and service you want to run strictly depends on MySQL. Do you want to set up another RDBMS? Of course no! You already know how to back up data for postgres, know it's cli, and now you need to lear mysql cli to set up back ups and it will take another amount of cpu and memory and all other consequences.

3 – A “setup” command that does most of hard job of configuring service. Or some single-entry item, which will take most of automatable actions done, like people do things in Makefiles.

It will hide all hard steps of dealing with stuff, so any non-techie-dude will be able to figure out how to do this weird thing in the terminal.

However, things are not such bad. There is lot's of services that are simple and shine easy to set up. For instance, I use RSS feed reader named Miniflux – it is simply awesome. The only thing I had to configure by myself was postgresql, just to create databases. And it takes really small amount of resources, so I can run it even on my phone if I want to. I just need to build it for arm64 architecture.

Also, there are organizations that create platforms to self-host services in one single click. They do the great job making things easier for developers of services, because they can focus on developing application instead of making it more “perfect” in perfection sense.

This was @alexcleac, I occasionally write things somewhere. You can follow me on mastodon or write as platform. Feel free to respond me if you have some thoughts on topic, but don't be rude, please :D

Have a good day :D