simone-robutti

In the bubble, a bunch of Tech Bros burn their brains to produce useless or dangerous data products designed on ridiculous premises, victims of what is usually called “dataism”. Dataism has many faces, being a complex ideology often devolving into superstition or religious fanaticism, but we can summarize it as the belief that data, its collection and circulation produce unbiased knowledge about an objective truth. The Human, the scientist, the analyst, the theorist become useless: machine can replace the production of knowledge and become the ultimate source of truth.

Researchers are stumbling again into old forms of pseudoscience , falling for the same mistakes made by scientists in the 18th and 19th century. Economic incentives towards novelty at all costs create monsters that appear grotesque to anybody living outside that specific bubble. The total absence of strategy and narrative about how to build a collective future that doesn't look like the present keeps the best mind of our generation busy trying to spot friendly patterns on their screens.

On the other side of the fence there are countless journalists, activists, politicians, researchers and in general the public sphere. These people are trying to stop this crazy suicidal machine that turns data into compelling stories and stories into profit.

Clearly, at the global level, the forces driving this phenomenon are quite clear: the immense political and economic power of the main offenders (Facebook, Google, Amazon, Netflix, ...) protect a system in which the proliferation of bullshit for profit is possible. Systemic feedback rewards the production of the useless. These big trees create the shade in which an ecosystem of services and products can grow undisturbed by regulators, protestors and cultural resistance. Rivers of ink have been spent on this topic and I don't have much to add right now.

But what about the daily experience of people involved in this system? What about your local tech hub, your meetups, the companies you find on StackOverflow Jobs and apply to? How does this surreal process looks like on the ground and from the inside? I think that as a Machine Learning Engineer I can give you some perspective.

The three types of Tech Workers in the Data World

The disciplines and roles in developing data products are always a complicated topic and there's no agreement on what these names mean: data scientists, data analyst, data engineer, machine learning engineer, MLOps and so on and so forth. For the sake of our argument though, this division is not that relevant. Your contribution to bullshit as a technical person is not necessarily determined by your skillset, but by how you’re positioned in the system or organization producing such bullshit.

That's why I believe it's fundamental to identify categories that translate the macro system described before into realities and individual experiences at the micro level. You might more easily relate to them, understand in which one you fit in and eventually decide to change something in your life or career.

I believe there are two relevant dimensions to this analysis: the soundness of the thinking behind the product and the awareness of how sound or unsound it is. Therefore we identify three categories: the product is Sound and you’re Aware (SA), the product is Bullshit and you’re Aware (BA), the product is Bullshit and you’re Unaware.

In this categorization I want to include mostly individual workers but, more often than not, they operate in companies, departments, teams and organizations that are similarly positioned and therefore it’s not unlikely that the behavior and problems of a type BA worker are shared by its own department or company. We could therefore speak about BA workers in BA companies, where the network of individuals expresses similar traits to the individual and vice versa.

Where do you stand?

Let's go more in detail:

Type SA

These workers know that their product is Sound and they are Aware of it. They know because they have strong, direct, unambiguous evidence (or if they are lucky, formal definitions) of the impact they have. This doesn't mean their product has a positive impact: a good part of these people produce software that is extremely harmful to society. Example: data-driven UI/UX designers at Facebook, that use data to maximize the engagement on the platform and to turn it into clicks. Their metric (engagement) can be tracked through experimental methodologies (like A/B testing) and it's mostly dependent by elements that are defined inside the system (i.e. the engagement doesn't need proxies, it's something that Facebook itself defines). Producing clicks is their mission and produced clicks is what they see.

Another example: the company I work for, Teraki. Luckily, they are not evil. They do data compression using Machine Learning on embedded devices (cars, drones, delivery robots and so on). Our metric is the size of the data that gets transferred : if what we put on the network is consistently smaller than traditional algorithms, it's working. Otherwise it's not working. There are no proxies, no ambiguities: the formal definition of the size of a binary sequence can be specified on paper in a few minutes and it's very easy to implement a reliable tracking system for this metric.

These workers, while not necessarily capable of framing their activities in a broader social and economic context, still find meaningful engagement in the problem they are trying to solve. They might narrow down the perspective in order to focus exclusively on the metrics that are a reliable testimony of their impact, but nonetheless be able to root their actions in a shared reality.

Type BA

Their product is Bullshit and they are Aware of it. This is the most common category in terms of sheer numbers, especially in middle or small sized companies. Though in some fields, they are the majority in corporate environments too. Type BA workers are the most interesting to observe because their work is extremely peculiar and fundamental to building the castle of delusions that we collectively buy into and call “big data” or “data science”. Small bricks made of tiny lies, half-truths and omitted details are built and placed everyday. These form thick walls and shiny towers made of investor cash flow, designed to impress the external observer and to hide from the gaze an uncomfortable evidence: knowledge is in another castle.

Type BAs know that what they are doing is pointless, because often they have enough background in Statistics, Epistemology or Scientific Methodology. These tools are often sufficient to trigger some reflection over the meaning of the button smashing; they know it's pointless but they also know it's profitable and therefore they keep doing it. The whole endeavour, though, is critically dependent on one thing alone: the person that pays for that work needs to hold a strong belief in what is being delivered, be it the manager, the customer or the investor. When unable to articulate to themselves how the economic and social incentives play into the epistemic inconsistencies, they dismiss their own work as an anomaly in an otherwise well-functioning system.

Their work resembles more the one of a graphic designer: it's not important if your logo effectively drives up sales, it's important that everybody is convinced that it's a good logo and therefore it will eventually drive up sales. A minute,yet critical, difference. The same is true for a model: it's not important if you can show rigorous, unambiguous metrics picked from established research in your domain, what’s important is that you can show metrics that can convince people. These people you are paid to convince might be your managers and you have to explain your work, lead them where you want, connect what you did with increased revenue and justify your role in the company. Or perhaps the managers are complicit: they know the product doesn’t deliver the promised impact. Then the people you need to convince are your customers. It doesn't make much of a difference: half of your day will be spent doing data science, the other half massaging the numbers, inventing stories and narratives, developing fancy presentations with plenty of graphs and tables.

It's easy to find in jobs descriptions for these positions lines like: “you need to be able to explain and narrate your work”. Sometimes they call it “data storytelling”. While explaining a complex data analysis is a fundamental skill for any serious scientist, most companies are actually looking for people capable and willing to participate in this bullshit-production process. This is equally built on effective production of insights and on systems of smoke and mirrors. Complacency is not enough to be qualified for such a job: the candidate needs a technical background as solid as any other and, on top, they should be able to use it in conjunction with interpersonal and communication skills to concoct the perfect narrative, capable of going through the defense mechanisms of the target receiver with a masterful use of statistics, metrics and other kinds of lies.

Clearly you cannot say the quiet part out loud before you start the job; it's whispered months later, in meeting rooms, when you are fully included in the company and have a stake in its success. Bullshit is your daily life already and only then it can be acknowledged. Some people decide to stay because they have to pay the bills, some others buy enthusiastically into it and embrace the mindset portraying the management or the customers as ignorant to deceive. They feel that most of the IT industry produces bullshit anyway and they tell themselves: “why should I move somewhere else?” Relevant and meaningful work is a privilege we cannot aspire to.

Type BU

They are the most dangerous ones. Their product is Bullshit and they are Unaware. These are the people that really believe they are saving the world despite everybody constantly yelling to stop what they're doing. They think:“They are clearly not talking about me, just Facebook and Google are greedy, I'm not like them”. Critical self-reflection is not an option: their world, their beliefs, their religion do not allow them to distance themselves from the product of their work and observe it from the perspective of the critics or the victims. They think Technology (with the capital T) is intrinsically liberating, good, beneficial and they are their loyal servants. Questioning this truth is beyond their possibilities because everything around them is reinforcing this self-deception: they are like flies flying repeatedly into a window. They repeat the same mistakes over and over, the same of the people that came before them, because they lack the means to understand and articulate what a glass window is and why their continuous movement is not bringing them anywhere. They believe their nonsense is an attitude of progress, an attitude of being on the bleeding edge, necessary to divine what the market really wants. The very rare success of one of their peers justifies in hindsight any waste of resources. The market becomes an optimization algorithm towards a global optimum and this requires exploring seemingly irrelevant paths. This magical thinking is rooted in the capitalist realism that dominates our times. It somehow connects the systemic sacrifice of mind and money to produce a few champions, capable of lifting themselves above the masses, and save us all through the achievement of the most important goal for humanity: profitability, the true judge that divides the worthy from the unworthy. Crazy is the new smart and we are still waiting for the supposed shared benefits.

This spirit, this social bubble capable of nurturing generations of worshippers of shallow criticism and enemies of radical criticism, of creators unable to understand their position in the world, attracts and unleashes an insane amount of energy. This comes in the form of skilled engineers, fervent entrepreneurs, huge investments from the Digital Capital and sometimes even big revenues, when the system becomes so huge that a vast majority of the public starts buying into the delusion. This world is also the source of the most ridiculous and surreal products that we often make fun of. Useless, toxic, dangerous machine learning or data-driven applications and research are no different; at best the issues are harder to spot for the layperson.

Where to go from here

Regardless of the type of worker you are, you're still involved in a very dysfunctional system. Tech has failed to meet its promises at large and while it produced some benefits for everybody and many profit for the elites, it did it at prohibitive costs in terms of human, social, economical and environmental resources.

The western Tech Industry is aggravating many structural challenges that, in face of an approaching economic and social collapse cannot go unsolved. Keeping the best minds of our generation busy with the creation of useless gimmicks or nefarious contraptions to maximize the number of impressions on ads is already a crime in such critical times.

To defuse the dangers brought by this bubble, we have to challenge the ideological and pseudo-religious premises of the whole endeavour:

  • Technology cannot solve everything:some problems are better solved through political and social effort.
  • Technology is not intrinsically good or useful. Even if the market seems to say so.
  • A system can be big, pervasive, naturalized and still be a superstition. Astrology and divination led the decisions of the most important people on Earth for millenia. The Enlightenment just skewed our preference towards superstitions that resemble science: our elites are not smarter than a Chinese Emperor or a Roman General. Most “Big Data” is just well-designed astrology, incapable of producing knowledge.
  • Slow, incremental change cannot tackle every challenge: sometimes change must be radical. A self-regulating system will always fail as soon as some new element is not registered as a feedback. Today we have plenty of feedback signals that go unregistered by the Tech Industry: ecological costs, social unrest, drainage of mental energy and so on.

On a personal level, the best thing you can do to avoid being part of the problem is to be critical about the problems you tackle and the solutions you propose. As a Tech Worker you're in a special position nowadays: you're among the people that have the tools to build a new world, but the old world deems you to valuable for its own reproduction and feeds you an idea of future that looks exactly like the present, but with flying cars and with some other useless gadget.

Being critical is not just about thinking very hard about what you're doing: it means creating a richer context for your decisions, reading about politics, philosophy, sociology, ecology and understanding where you sit in the world and what impact will your work make. Ask yourself questions: what is my working changing in the world? Can I measure it? Am I just measuring proxies of proxies? Who benefits from these changes? Why? Am I really convinced of what I'm doing? Am I just trying to find a problem for my solution? Is the problem I want to solve even relevant outside my bubble? How small is my bubble? Why is the market or the investors rewarding me even if I don't seem to do anything meaningful? In the end, should I just give up and go do something else? Be honest first with yourself, then with others.

Technology, including Machine Learning and data-driven approaches can be useful. It must be applied or designed in a useful way because there are great challenges in front of us. This is not something a single person can solve and it's not something you can solve inside the startup world. Be critical, be skeptical, reject everything around you if you believe it's tainted by this new religion of data and code. The next generations are asking you to. History is asking you to. In the face of a dying ecosystem, rising authoritarian governments and the disintegration of the social fabric, technologists cannot waste time. There is no alternative.

This article is part of a series which will hopefully lead to the creation of an open article about technological dissemination and the narrative that surrounds how it’s understood and performed in the western world. The articles can be read independently from one another or as a collective whole.


Prerequisite readings:

The Californian Ideology


The word “hacker” has been employed by so many people and movements that it is now devoid of meaning. While its origins as a term initially began as a way to define a small group of tech enthusiasts, it slowly evolved to identify a subculture which then grew into a constellation of political movements. Nowadays, the term “hacker” has trickled onwards into mainstream startup culture. It has become a term used by technical employees and business people to identify themselves with an image of success in line with the Californian Ideology. The entrepreneurial effort is then portrayed as a challenge against the status quo, the common sense and the boundaries that constrained their competitors (the “box” the hacker can think outside of). Hackerism became an element of marketing competition.

Nowadays the term is used in mainstream media mostly to identify security hackers or misused to identify professional or non-professional crackers. The term has been appropriated by many and diverse actors in the technological, social and political landscape to the point where the understanding of the word has to deal with apparently irreconcilable contradictions . What does a “growth hacker” (basically a business strategist without a suit) have to do with an hacktivist trying to bring neoliberal capitalism to its demise by attacking banks or by developing a messaging app? Are they kin to a Chinese security expert trying to disrupt the infrastructure of some American company?

“NO!” shouts the hacktivist: “I'm the only one entitled to this word, because being a hacker means to be against the system”. The developer at a startup would nonchalantly answer: “Brother, I'm also against the system. My company is trying to disrupt the market of shower filters. For too long this niche has been dominated by a cartel of old capitalists that never tried to improve their products”. The Chinese hacker wouldn't participate in the discussion at all because he doesn't marry in any of these ideologies and he just inherited a label from the Western discourse while trying to earn a salary.

There are many categories which self-identify as hackers but the goal of this article is to talk about a specific subset of them that we will now try to define. Some might call them “hacktivist”, but it's still too broad of a term. We are concerned with those hackers that perceive themselves as politically active, politically conscious and leaning towards progressive positions which are intended simply as promoting an expansion of natural or artificial rights to a broader spectrum of people. Among these, we consider only those that work actively to analyze and probe existing techno/social systems and artifacts or to develop new ones. We exclude from this discussion those whom are not politicized and those whom are leaning towards reactionary, pro-capitalist and racist positions. We also exclude all those political crackers (individuals and collectives) that in the last few decades attacked corporate and governmental systems to extract and publish sensitive information that they believed belonged to the public space, such as Phineas Fisher.

The hackerist perspective

The “Hacker Identity”, I argue, is a powerful source of motivation. It is a catalyst for the construction of technology outside the mainstream pipelines of technological development. I also want to argue that this identity comes attached with a heavy ideological and practical baggage that ultimately hinders the contribution given by “hacked technology” to the liberation of people and improvement of their material, psychological and spiritual conditions, an achievement that a specific subset of hackers would like to claim as a justification for their activities.

Let’s define the “hackerist perspective”:

The hackerist perspective is an attempt to alter technology for political reasons by repurposing technological artifacts without concerning oneself with altering the process that produced said technology.

Follows as a corollary: The process and the systems that produce technologies are questioned, attacked, controlled but never repurposed.

The hackerist perspective is by this definition anti-political, because redefining processes and systems requires political work towards consensus, regardless of the scale. A given way of using a technology is encoded by the social structure that employs it. Such coded usage cannot be then embedded into the technological artifact without first paving the path on a social level. The hackerist perspective cannot encompass such complexity: to retain meaning for the efforts made by the hackerist multitude, it resorts to flip the myths of the Californian Ideology upside down. Where the Silicon Valley imbues its technology with saving powers, the hacker sees a Dark God of Control. New, pure, moral techno-Gods created by the hacker will liberate the humans from the oppression of the old, tainted, evil techno-demons created by the corporation.

Narrowing down the action to the technical level and erasing the social and ideological implications of said operation is the only allowed option after embracing a worldview that prescribes the struggle for freedom as a clash of technical capabilities. A machista confrontation of individual or collective intellects rather than a fluid struggle of human communities and bodies against the capital.

The hackerist perspective in the real world

How does the hackerist perspective influence the world around us? How does it influence the Tech counter-cultural community? How does it influences the artifacts that are produced? Is it just an empty category or a category useful to escape ideological cages?

Being a liquid identity and lacking clear structure and organization, the “hacker community” is hard to narrow down to a specific set of people, goals and projects so we must proceed with nuanced categories.

Let's start from a fact: a huge number of activists around the world are concerned with resisting the negative impact of technology on our lives and to limit the ever-increasing power of the corporate world over societies, individuals, organic masses and spaces all over the world. A good part of them would identify themselves as hackers or hacktivists. To this multitude of perspectives, judged as a single entity, can be attributed successes or failures in the many goals that seems to drive them.

The overall balance of the hackerist struggle is deeply negative. Today, the hacker multitude is incapable of producing pervasive alternatives, producing viable and accessible solutions, and ultimately counter-attacking the continuous invasion of Big Tech of the public and private spaces. Like most of the post-'68 left, hackers are content with resisting the onslaught, slowing down the forces of capitalism operating at a level that is not reached by the forces of the Resistance. The result is the creation of small spaces of safety which get harder and harder to defend, requiring increasingly larger efforts to be maintained.

Let’s take private and secure communication over the Internet as an example. The multitude created many viable and usable solutions in the last few years to allow most tech-savvy people to exchange text and files in a relatively safe way without having to invest too much time in configuring their systems. While this result is highly effective in protecting members of the multitude and other actors like journalists, whistleblowers and political dissidents, it seems to be unable to be generalized to the masses. Probably most hackers would say that this was not even a goal to begin with. The pattern of prefigurative localism collapsing into self-referential systems of value is the same that can be easily found in many niches of the political left.

The inability to generalize and scale their solutions (with the exception of a few Free Software programs and some hardware designs like Arduino) seems to be a common pattern in all the “technological confrontations” that the multitude engages in. A change that is restricted to an élite chaste of tech-savvy people is neither lasting nor impactful change. Reproducing their individual independence from Big Tech over and over is not a fight for the liberation of humanity. In the same way, work on philosophical theory that fails to produce results in the real world is nothing more than a masturbatory act performed by an entitled middle class of academics, artists and cultural workers. The parallelism between these two worlds, the hacker multitude and the left-wing cultural and political multitude is uncanny, even though the two forms of functional immobility come from very different roots.

The hackerist perspective is enough to explain this limitation: there's just so much you can achieve by concerning yourself exclusively with operating on a technical level. On top of this, there seems to be a sequence of trending technical solutions to which hackers attribute liberating power: free software, decentralized solutions, federated solutions, encryption, client-side computation for data ownership and so on.

These days federated solutions seem to be trendy, with lot of hope put into Mastodon, a federated clone of Twitter that replicates the UI/UX of a software designed to extract data from users and make them engage in a rapid, toxic, and confrontational way to maximize engagement and attention. Most of these design choices seem to go unchallenged. But it's not much of an issue since most instances are populated either by LGBTQ+ communities, left-wing activists or hacker/tech-savvy people. And also fascists: neo-nazis, white supremacists, right-wing libertarians and so on. Since they are not allowed to organize on commercial platforms, they make use of every technology that can give them independence from the control systems of liberal societies. A scenario for which Mastodon seems to have no effective countermeasure, and a consequence which wasn't envisioned beforehand. As usual, the creator of the technology won't be held responsible for the outcomes of providing a tool to Nazis. On top of this, there are serious doubts that the federated nature of the social is actually delivering a distribution of power among many stakeholders: the relevant instances are just a few and the distribution is, so far, extremely skewed.

The hackerist perspective vs the holistic perspective: Mastodon vs FairBnB

Mastodon represents a good case study of the naiveté of the hackerist perspective. To understand why the hackerist perspective is problematic, I think it's useful to point at another case of software aimed at liberation that seem to approach the problem from a deeply different perspective: FairBnB.

FairBnB is an attempt at creating a non-exploitative alternative to AirBnB. It does so by employing a more fair business model, investing in local communities, respecting data ownership and trying to build a worker/user run platform as a cooperative effort. Being a new and still not open platform, a judgement on the outcome is not possible but that's not why this is interesting to us.

Mastodon and FairBnB lie at opposite ends of the spectrum of technological liberation. While Mastodon begins its endeavour with a technical question, FairBnB begins with a political and social question. “Can we make Twitter federated to liberate its users?” vs. “Can we liberate cities from the harm brought by mass tourism?”. Once we answer to the second question, giving it a technological embodiment is relatively easy. The converse though is not true: once we have a technological artifact like Mastodon thrown out into the world, it is impossible to shape its politics. A deep political reflection on the liberating power of Mastodon would probably invalidate all the efforts made so far: such an approach would never be employed by the lead developer and the community.

It is no accident that these two software artifacts differ so greatly in intent and implementation: they come from very different communities with contrasting ideologies. Mastodon, being an exemplification of the hackerist perspective, puts its trust in the liberating power of the technology itself as we have discussed before. In contrast, FairBnB comes from a more traditional analysis of power dynamics in local and global economics, a critique of platform capitalism and a desire for communal services, all of which are characteristic traits of a leftist environment that is still struggling with its relationship with digital technology. Nonetheless, the idea that technological repurposing can be employed as a weapon against the Capital is growing stronger and stronger among said leftists: FairBnB is one among examples of this renewed alliance betwen the Left and Technology.

End of part 1. Part 2 containing a pars construens will follow.

A friend of mine, one day, came to me and said: “Read this book, it's the Das Kapital of our century”. Since he reads a lot and I trust his judgement, I picked it up but with a lot of skepticism because its author, Peter Joseph, was associated with plenty of populists, conspirancy-oriented movements thanks to its previous works and its Zeitgeist Movement. I didn't really consumed much of its content back in those days so I decided to give him a chance.

Little spoiler: the book is not the new Das Kapital but it's still worth the read, especially if you are not really well-versed in sociology, political science and economics. While heavily sacrificing rigour and simplying a lot of topics for the sake of clarity and contrary to my expectation, the book is not a pile of pseudo-intellectual arguments to attack a non-well defined reptilian enemy somewhere in Washington.

The goal of the book is, in fact, to bridge the gap between the beliefs we hold in the western society and the last century of scientific results. In the first pages Peter Joseph states clearly how our culture, our values, our social structure and our institutions have failed to keep up with the progress achieved in many fields like sociology, economics, biology and psicology. The result is a stark inconsistency between what we hold as “scientifically valid” and how we structure our social and political action.

In each chapter he builds an argument to challenge the status quo and invite us to rethink our outdated institutions in order to address a central problem with multi-faceted materializations: systemic violence. For him, systemic violence is an emergent property of a system that brings damage and oppression upon individuals and social categories. A form of violence where there are no perpetrators to be punished, no bodies to put in jail but only rules to be changed and structures to be evolved. The effects of this kind of violence are all around us and have been analyzed in detail. Its complexity though makes it hard for academics and fringe politicians to include it in the mainstream political discourse, especially in an age of simple answers to complex problems.

Nonetheless, employing a systemic perspective is paramount to improving our society, limit the impact of the approaching ecologic and social collapse and ultimately claim to act morally. The book is, at its core, a series of well-elaborated exaplanations of systemic problems from ground evidences to moral and political conclusions, through the lenses of different fields. Examples includes an analysis of how poverty has causal effects on health, life expectancy and self-realization in life, an anthropological discussion on how we value work as a necessary punishment, how ideology and faith determines macro-economics decisions and so on.

The book is then wrapped up by the last chapter that summarizes the ethos of the de-growth and xenoaccelerationist left: building a new future, rethinking the necessity of work, rethinking monetary systems, restructure the system to be free from the necessity of growth, do politics for what will come after the collapse.

So far the book could have been a 10 out of 10: we said it's solid, it touches many interconnected arguments without fear of dealing with them in combination, it's rooted in daily experiences of suffering for many people and it's topped with a pars construens. Unfortunately the execution is not as good as it could have been. I still believe this is a book you should give to your liberal friend to radicalize him: it's a good tool, not necessarily a good piece of writing.

The main problem I have with the book is that it's continously quoting academics and this gives the book an aura of authority but these quotes are often out of context and seem instrumental to reinforcing the author's argument, more than investigating any sort of truth. The boundary between the two endeavours (questioning the status quo and asserting scientific truth) are often blurred and it's impossible for the reader to follow these strains of thought. This would be ok if he carefully selected the authors to quote, but among them we find many controversial figures such as Vandana Shiva and Jared Diamond. He also quotes David Graeber “Bullshit jobs” that while being a good read, is by no mean a solid piece of writing.

Any skeptical reader's trust in the author will be often challenged. I clearly don't believe this approach has been used in bad faith and it's clear how covering so many diverse and sensitive topics for a single person is a titanic work where you want to have your back guarded by as many experts as possible. This is particularly evident in the second chapter, in my opinion the only bad one in the book, where the anthropological arguments, behavioral and economic arguments are poorly stitched together. Nonetheless there's room for iterative improvements that I hope to see in future editions.

In conclusion, the book is a valid instrument to circulate many ideas that are too often presented in an inaccessible way. It's written for a target that is still entrapped in many ideological cages that the author assume are still in place for the reader, contrary to most of the material he references, that often assume a certain degree of ideological alignment in the reader. The accessibility is the most precious trait of the book but the price paid for it might have been too high.

I haven't been writing videogames' reviews in many years now and I'm not changing my mind, but I want to talk about a videogame anyway: Mindustry. You can buy it for 5€ on Steam or just download a .jar file from Github and run it.

screen1

The game in itself is well thought, well designed and well executed. I don't want to go in detail about the merit of the game itself, but let's try to understand what it is about. It falls in the category of automation games, like for example Factorio, but it has much more prevalent Tower Defense elements. The player can dig up (infinite) resources, move them around and combine them in increasingly more complex chains through factories and different transporation methods. These resources can ultimately be fed to many different kinds of towers that will help you defend from waves of enemies, together with bots that can be created in dedicated factories. While these basic mechanics are just few and simple, Mindustry gives it a very horizontal width by allowing the player to build a big number of towers, fuel them in many different ways through a diverse set of production chains.

While the premise looks similar to Factorio (minus the ecologist message), the gaming experience is very different: Mindustry is faster, more multiplayer/PVP oriented and more open to personalized strategies. It is challenging without being (too) frustrating. In campaign mode, it introduces you to concepts gradually and leaves you enough time to experiment with them without being unnecessarily slow or repetitive. The sessions, both in single player and multiplayer, are shorter and more influenced by the specific layout of the map. Again, if you liked Factorio, you will like Mindustry but not because it's a “more of the same”.

screen2

What I want to talk about is something else. This game delivers a mature, cohesive, polished visual and gaming experience that, in my opinion, is rarely matched in an open-source game and the developer(s) should be commended for it. If you have been involved in an open source game development, you know how hard it is to give artistic and design coherence to the work of multiple developers. Mindustry, for what I know, took a different path: it is the product of mainly one developer, Anuken that open-sourced the game while retaining ownership of the development.

This is his main project and he took a rather bold strategy to promote it compared to most indie developers that follow more traditional monetization strategies. Nonetheless the game fits this model perfectly well: it's small enough, it has many orthogonal elements of gameplay, it has serendipity and small mods give a lot of added value. There are clear sinergies between its Open-Source nature and the design of the game.

Contrary to many other open source games that aim at reproducing commercial games (like 0 A.D. or FreeCiv) or others that target micro-niches of hardcore gamers (like Cataclysm: Dark Days Ahead), Mindustry has the ambition to create something new that caters to a wider audience. It delivers a real and proper game and not a toolbox to build your own gaming experience like too many niche games are doing. Not that this is bad, I've been a hardcore Dwarf Fortress fan for years and I loved games like Cataclysm or Aurora 4X, but it's now time for open source games to target a different crowd.

On top of this, the whole process of contribution and release is very well structured and trasparent. The documentation is ok and it's very easy to enter the development process. The game is developed in Java with LibGDX and on a technical level shouldn't be challenging to contribute to. I've used the library in the past without previous knowledge on gaming development and I managed to achieve very good results in a relatively short span of time.

editor

The project is now just two years old and just landed on Steam, but as I said, it's already an extremely enjoyable experience as it is and has a very active community around it, with populated online servers. Is the game enough to start a new page of Open Source gaming? Probably not. Nonetheless is a small pearl that everybody in that world should consider as a blueprint to open up their projects. I hope that in the future the main developer will document better his subjective experience and give tools to other developers to emulate him.

When you first learn Python, somebody explains you that you can add the folder in which your source files sit to the PYTHONPATH environment variable and this code will then be importable from other locations. Too often this person forgets to say that this is a very bad idea in most scenarios. Some people discover this on the internet, some others just realize this by themselves. Too many people (especially non-programmers) just believe there's no alternative.

This blog post is for all of them, because even if you know that an alternative exists, it's not always easy to grasp. Python tooling is confusing because there are many software, built on top of each other, with a lot of overlaps with their concerns. It's hard to understand how they fit in the big picture.

For this reason I've decided to create a post that lists the most important tools, when and why they are used and what problem they solve. I will try to explain with simple words how you should approach each one of these tools. If a tool is here, it means that, as a Python programmer, you're supposed to at least know its existence. I will list only tools that can be applied to any project or workflow and that you should consider every time you start a new project. This doesn't mean you always have to use each one of them on every single project. Too much tooling can be easily be an overkill and become hard to maintain in some cases.

Basic

Setuptools

Setuptools is the standard way to create packages in Python. It's everywhere, it works and it fulfill its role.

For: building egg, zip or wheel files from source, define metadata for your project, share code in a structured and standardized way When: basically every time you want to write code that should run on somebody's else machine

Alternatives: Poetry, Flit

virtualenv

Virtualenv is a virtual environment manager. These isolated environments can be understood as self-contained python installations with independent packages installed. Using virtualenv means that you don't need to (and you shouldn't anyway) install packages using your default system's python installation.

For: keeping your dependencies separate, supporting multiple python versions in the same system, moving dependencies around easily When: you want to develop code, when you want to use a python version different than your default without going crazy

Alternatives: Docker or equivalent

Pip

Pip is the most common package management tool for Python. It allows you to take local or remote packages and install them in your virtual environment or system's Python.

For: installing and uninstalling packages, tracking versions of the packages you're using When: always

Alternatives: Poetry, Conda

Packaging and Distribution

For a more detailed overview, python.org has a dedicated [page].(https://packaging.python.org/).

distutils

distutils is a precursor of setuptools. The latter makes heavy use of features from distutils so it's not uncommon to have to interact with this tool. It's not something you should pick for your tool belt directly but you should be aware of where it fits in the big picture.

Pypi

Pypi is the Python Package Index. It's a big repository of all your favourite Python Modules. Pip, for example, takes built packages from here.

For: publishing your code When: when you have a package that you want to make publicly available

Pypiserver

Pypiserver is one implementation of the Package Index API used by Pypi. You can set up your own repository, for example for your whole company and publish packages there without releasing them to the public.

For: sharing code inside an organization When: this code shouldn't go public and you want to have control Alternatives: Warehouse (the one used by Pypi), djangopypi

Poetry

Poetry is an alternative packaging system that replaces setuptools, pip and some of the tools built on top of them. It's an attempt to do a complete overhaul of how the Python packaging system works. So far it got some traction and lot of positive feedback but it's far from being the prevalent option.

For: handling and distributing packages, managing your dependencies, avoiding dependency resolution problems When: you have a fresh project and you're not afraid to use a relatively niche tool Alternatives: Pipenv

Pipenv

Pipenv, like Poetry, is a tool to structure your Python project dependencies and configurations in a more sane way. Through a Pipfile, it manages the dependencies for your project and ensure consistency and ease of use.

For: handling and distributing packages, managing your dependencies When: you want something like Poetry that will raise less questions Alternatives: Poetry

Documentation

Sphinx

Sphinx is a tool to build documentation. It was born originally to handle Python's documentation but has now graduated to a general purpose documentation tool. It remains the most common option for Python Projects.

For: producing PDF or HTML documents from reStructuredText sources When: you want to have external documentation for your project, your APIs and your code Alternatives: Docutils, Doxygen

autodoc

autodoc is a fundamental extension to Sphinx that allows you to generate restructuredText files from Python source code with entries for each class, function, module and so on.

For: documenting your code or APIs When: probably every time you're using Sphinx for a project Alternatives: autosummary

Testing

py.test

py.test is, in my opinion, the best test suite available in Python. It has plenty of features even though not all of them are presented properly so it takes some time to discover all the many possibilities supported by this software

For: testing your code When: always, you lazy ass Alternatives: unittest, nose

Hypothesis

Hypothesis is a tool for property-based testing. Briefly said, it generates random test scenarios according to your specifications until it finds a case that makes your test fails. Take some time to learn the principles behind before starting to use this tool.

For: testing code, especially data processing When: you need to test non-trivial logic with a wide input space (numbers, strings, structured data)

tox

tox is at its core a virtualenv manager for testing. It means that you can configure it to run your tests in a series of clean, customizable virtual environments to ensure that your code will be able to work under different conditions. All of this without any manual work required.

For: code that needs to run in different conditions and environments. Also useful for CI. When: your code needs to support different Python versions, run in different environments and in different operative systems Alternatives: bash scrips, CI pipelines

Other

pyenv

pyenv is a python version manager. It aims to simplify the local workflow of developers when handling multiple versions.

For: running different projects with different python versions When: you need to run with global python versions and you have many Alternatives: manual management, virtualenv, Poetry, Pipenv

PyScaffold

PyScaffold is a tool to initialize your project structure in a standardized way and to provide some of the tools we listed before without need to configure them manually. It's heavily customizable.

For: bootstrapping projects, have multiple projects with uniform tooling and structure When: always (as long as you know the tool, don't use it the first time if you're in a rush) Alternatives: python-project-template, Cookiecutter

flake8

flake8 is one of the most used linters for Python. It runs different scripts to verify the compliance of your code with Python's style guide requirements (PEP-8).

For: verifying and guaranteeing good code style in your project When every time your project needs to be read by somebody, including yourself Alternatives: pylint

Black

Black is an automatic code formatter. It means that instead of just checking your code for compliance, Black will actually modify it to make it compliant.

For: formatting your code automatically When: you have no problem giving up manual control over your code look Alternatives: autopep8, yapf,

edit: I'm compelled to specify that this post is not to be taken seriously, it's satire if you want and most of the statements are ironical in nature. Don't take it literally.

Inspired by this post I've decided to write a post about the worst programming Language Names.

Programmers are often notoriously bad at naming things and sometimes managers help them too. This produced many specimens of badly named languages that made it harder to use and discuss them. Here are the ones I dislike the most.

SQL

How is it pronounced? Sequel or S.Q.L? And then a Sequel to what? It's too old, all those remember are probably dead or working as clerks now. Don't get me started with all the problems it creates with PostsgreSQL. Is it Postgres or Postgre?

Go

Go. Go where? And why? How is a novice supposed to Google it? Should we just call it Golang? It sounds too Oriental.

C

Can you at least make an effort and come up with a real name? It's not a name, it's a letter. If I call a variable “c” everybody gets mad, but apparently it's fine to call a language “C”.

Scala

Scala is a shortening of Scalable. Fair enough, that's clear. But in Italian it also means stairs. Therefore the logo is the staircase of an EPFL building, where Scala has been developed. The problem is that stairs are not really scalable. Also Teatro alla Scala is a very famous Opera House in Milan and it happened at least once that a lady came to the Milan Scala Meetup with a little kid, expecting it to be about Opera or something like that. They sat through the whole talk. It was something about monads. It's always about monads. Well, they should have picked a less ambiguous name.

Pony

I've recently entered the community and started to learn the language. I still sometimes forget to write “Ponylang” and Google keeps reminding me that the target audience of My Little Pony is single guys with hygiene problems. I just want to share mutable state between my actors without facing the horrors produced by modern society. Sean, I'm disappointed.

Enter your email to subscribe to updates.