neverstew

In David Epstein's book, Range, he outlines the hazards that sticking to the congruence doctrine oft-taught in business schools can surface. Tech companies are just as susceptible to this as any other, here I reflect on some ways that might encourage incongruence as part of our daily engineering practices.

Don't start with SCRUM

In the world of startup work that I inhabit, teams come and go quickly. They emerge to tackle challenges, change team members and focus quickly and are required to tackle uncertain problems.

Too often, I've seen teams that try to treat this uncertain world with the certainty that SCRUM offers. It's a tool that is good for some teams, but not all and more often than not, SCRUM leads to frustration that the guard rails and certainties that the team are forced to adopt don't help solve the messy and uncertain problems they're faced with.

SCRUM is a fairly sophisticated process when done well and requires that a team is firing on all cylinders towards the same goal. Teams need time to build up the trust in each other that this requires, as well as build their shared understanding of the problem they face and the goal they're heading towards. Give people the time and space to work these things out as a team by ditching SCRUM. Instead, go for a simple kanban or a list. These are tools that everyone understands and using them allows for the most important thing: flexibility. Incrementally adopting parts of SCRUM is a great way to introduce it as the team graduates towards focused delivery.

In the messy and uncertain world of startups, focusing on delivering functionality is a red herring; teams must be focused on unearthing the problems that people face and solutions that might help them get there. People need the space to change their mind about what is right when the conditions are uncertain, they need time to refine and adapt their understanding, they need a healthy level of incongruence to tease out the important pearls of wisdom hidden in a sea of noise.

Sense making, not decision making

To get the best out of a team, you need people with a set of diverse experiences. Those people must also be able to share their experiences in a maningful way. Often this experience will come in the form of criticism, with suggestions for improvements following shortly after. Sometimes only the critisicm will be available and the team must work together to understand how to get past the new problem.

As a leader of a team, one approach to making room for these improvements is to adopt a “sense making” role, rather than the more common “decision making” role.

As a decision maker, you become the limiting factor in understanding. For your team to progress with an idea, it has to go through your brain and be interpreted in a way unique to you. Even with the most exceptional mind on the planet, you create a subtle dynamic that prevents criticism. If a person raises an exception to your decision, it will be interpreted as a criticism of a personal action that you, the decision maker, made. This direct, confrontational criticism is something that many people find uncomfortable. Rather than offer criticism, people will avoid doing so in case it is interpreted as a personal slight.

In contrast, a sense makers role is to pool information from each of the sources available and distill it into a coherent plan. By abstracting the plan from the distillation of information, it becomes something shared by the team. People are free to criticise the plan or add new information to influence the final version without it being interpreted as directed at any person.

By taking up the role of the sense maker, you allow more contributions to the plans gonig forwards without sacrificing responsibility or the ability to rally a team around a specific version of the plan.

Code as an understanding of the world

A person smarter than I once described software (especially the Object-Oriented kind) as an encoding of understanding of the world. All of the logic we create and names we provide are in service of one thing: describing and codifying the rules that govern our slice of the universe. For the life of me, I cannot remember who first voiced this interpretation of code (if you can, please get in touch!), but this framing of the systems we build allows us to understand where we can fit in incongruence.

An important baseline before being able to contribute towards a system of any size is the ability to understand what the rules of the system are. This is true when taking any size view of the system, from minute detail to big-picture overviews.

In order to effectively find and contribute to flaws in our understanding of the world or the view of it we're building with our software, we must be able to understand it with the smallest cognitive load possible. By relieving our minds of the burden of understanding what the code is doing, we allow it to focus what it should be doing. This invites people to refine the understanding of the world that the system provides.

There are reams of articles about how best to do this, but one of the most practical demonstrations of how you might go about doing so can be found as part of the Elixir tutorial on building OTP applications.

By structuring your code clearly, in plain english, with logic and intent clearly defined and highlighted, you create a space that people can rally around and discuss changes to their shared understanding of the world. This is the heart of building incongruence into your team: creating a space for discussion of the facts as you collectively understand them.

Over the past year, I've found my attitude towards job titles has shifted. If you're like I was about a year ago, you might be largely neglecting trying to secure another job title. This post might convince you why job titles are important in the average workplace and why you should be paying more attention to them than you are. It will, at least, serve as a reminder for me in the future should I ever forget.

My problem with job titles was that they hide the things that you really want to know about a person. Are they good at their job? Do they value collaborating (with me)? Do they work hard? Are they going through a tough time outside of work? Do they have a family? Have they had other jobs before this? All these questions and so many more are what really amount to the experience you have interacting with another person. At their most friendly, job titles are reductionist. At their least friendly, they bolster the ego and patch over deficiencies with assumed competence.

“I shouldn't need that job title” was a phrase that commonly looped around my head. I wondered why, despite a lot of hard work, a lot of time helping others, a lot of energy spent understanding what motivated people tailoring our interactions around those values, my efforts weren't being recognised by others. Then, after a period of hard work and little to show for it, some pieces clicked that changed my perspective.

Everyone else is doing it

What a terrible reason to do anything.

This is, however, the core of my argument. It's a bit more nuanced than that, but I'm hopeful that you'll keep reading to find out why I would resort to a line of reasoning I proclaimed awful just a second ago.

We don't work in a vacuum

No person operates in a vacuum. Whether it's via an intra-team relationship, towards a boss, towards stakeholders or even with a customer; we all operate as part of a group.

Each of the people in your group(s) has a distinct identity; multi-faceted and with a completely unique set of experiences. You could probably describe the complex identities that some of your close team members have. Can you do it with your boss? Can you do it with their boss? Eventually, we run out of room to truly know people and their myriad experiences.

I was expecting people to understand my background though they had never seen it. I was expecting team members to somehow find out that I was good at something and that I would gladly share my time with them. I would wonder why another senior person might be asked a question that they would then defer to me, yet continue to be asked those kinds of questions by the same people.

We seek to understand our relationships with others

People are inherently social creatures. All of us exist within multiple social structures and occupy different roles within each of them. Having some sense of what our role is allows us to feel confident in our belonging.

These structures build up slowly over time. At work, they often start as rigid agreements between each team member about what their role looks like. These change over time as we get to know one another.

In each of these interactions, we want some idea of the role that the other person is occupying when they're talking to us. Is it a colleague, a friend, a superior, or something else entirely? Understanding this context is important to how we communicate and it can even change during a conversation.

Each of us might have the capability of fulfilling many different roles at different times. Add this to the complexity offered by personal experience and you've got a huge amount to take into account.

Making shortcuts

With all this information available, it's impossible to digest it all. People look for ways to distill the complexity into simpler representations. This could be through roles and duties, it can be through nicknames, it can be through reputation built up and discussed throughout the team over time.

One of the simplest ways for people to simplify all of the information is through a job title. A job title is a shortcut to understanding. People rely on them, especially in groups where they are new, in order to hone in on the right person(s) to help them.

Complex relationships and nuance trump a job title every time, but the road to understanding and a good relationship with someone is long and paved with ups and downs.

Everyone else is offering shortcuts. Do everyone a favour; offer people a useful shortcut. If there isn't the right shortcut for people to take, strive to make one.

This article attempts to distill some thoughts about Conway's Law and the resultant pros/cons at Residently. Only recently have I started to think a lot more about Conway's Law and how it might apply to the structures that I find myself a part of. This has mainly come about due to observations of pain points in the Residently structure that have arisen as the team has grown.

Conway's Law

Not everyone ascribes to it but I have found myself agreeing with the notion behind it. More importantly, I think, is to view it not as a constraint but as a description of the way in which we naturally, efficiently, understand and communicate about systems.

Software is inherently, I believe, a people problem. I don't work at the scale where micro-optimisations are the name of the game and probably never will. For me, software is about solving problems. In this sense the largest constraints are the understanding of problems and the process of ideation and testing the solutions you design. The name of the game here is to be able to build your system as easily as possible in any direction that might be required. Anything that gets in the way of iterating on the idea is preventing the team from progressing at its most fundamental level.

Enabling progress in small teams

With that frame of mind, Conway's Law would suggest that the best way to enable a team to iterate on ideas is to design our systems in line with our team structure.

This firmly suggests to me that microservices, multiple repositories and other forms of distributed modules within a system are a terrible idea for small teams. This conslusion resonates with what I see day-to-day and provides a nice explanation/guiding hand for future projects that I might undertake.

I wanted to write up some of what I've been working on in my spare time. I'm doing this for a number of reasons: part reflection, part record, part catharsis.

Earlier this year I read The Mind is Flat by Nick Chater. It is an absolutely fantastic book that completely changed my view of human consciousness. I would thoroughly recommend it to anyone interested in the human mind.

In The Mind is Flat, Chater talks about the differing capabilities of modern computational feats and the mind. These mostly boil down the mind being exceptional at large, imaginative connections that can access completely disparate portions of knowledge whilst computers excel at standardised tasks.

He sees future advances as enhancing this relationship, just as past advances in the agricultural and industrial revolution helped to move us on, but played on the strengths of each of the parts. I agree that this connection could be very valuable, and this leads me to think about some of the most useful cases that I can imagine.

One of the first connections my own mind made is the link to The Extended Mind concept. Could we use personal computers (i.e. phones and smart wearables) as an effective extended mind?

In many ways we already do. Wikipedia is an excellent example of this and can be accessed by pretty much anyone at any time, anywhere. Other productivity tools (especially GSuite and its contenders) allow us to look up information on the fly. What we lack, however, is the tools that enhance the relationship between computer and mind; instead choosing to replace functions that the mind is often poor at and act as a crutch. Can we, instead, use these computing tools to prompt and elevate the mind in a way that could produce even better results than it already can?

What would this look like?

One of the most amazing things that our minds can do is to draw connections between seemingly disparate ideas. Not just random connections, but connections with meaning.

After having a little think about this, it seemed clear that one way to enhance this function to form links between seemingly disparate sources of information was to make all of the sources readily available. Once the information was at hand, then the mind would have an easier time drawing the connecting lines between the points.

Initial Sketches

The first sketches here show the ideation process about the overall concept of how to surface other 'nodes' in the knowledge graph.

[insert sketch]

What is also clear about our brains is that the strength of connections between neurons is the path to greater understanding of a problem. Along these lines, another concept grew.

[insert sketch 2]

Tech Decisions

I wanted to try this out, so I had to build a prototype. There are some design principles that needed to be adhered to for it to work:

1. The graph must always be available for it to act as an extended mind
2. The interaction must be quick and reliable
3. The tool must ingest a lot of information for it to work well, so it must be both appealing to use frequently and scaleable

In order to meet these criteria, I ended up with a few key technology choices:

1. React Native for the user interface
2. FastAPI for the backend
3. neo4j for the database

These allow me to meet all of the design principles, as well as iterate fairly quickly. neo4j would be a new thing for me but that's interesting in itself.

Building the prototype

Building the first version took a few days of solid effort. But it now lives on my phone!

[insert screenshot]

You can see that the interface shows entries that you have made, along with tokens that represent the nodes of the graph (lemmatized versions of the input words). You can also easily add new entries in another screen.

The backend runs in a small droplet on DO for about $5/mo. Both the database and the server run in a docker-compose network fronted by traefik. This is cheap enough that it's not breaking the bank and easy enough to manage by simply SSHing into the droplet, pulling the latest images and restarting the containers.

A few weeks of use

After a few weeks of using this tool, I've learnt a lot about the shortcomings and successes of the format. Here's the graph as it stands after the first few weeks.

[insert screenshot]

Findings

It's hard to sanitise input

Boy, did I underestimate the effort of what is essentially an NLP problem. Parsing unstructured input into useful nodes in the graph is difficult. The more that I thought about useful atomic entities that have meaning as one node in the graph, the more complicated it got. Some simple examples here are: parsing numbers (with or without spaces, commas, decimals etc.), parsing URLs, parsing dates.

neo4j is not mature for python

I take for granted many of the database tools that I use in my day-to-day life of SQL transactions. They basically just work. neo4j with python is not like that. The client is decent but integrates with type hints poorly and has strange semantics for executing queries that still need to be worked out.

It takes a lot of input

It was clear that there needed to be a lot of input into the graph for it to be useful. I underestimated how much. Despite using it quite a lot for a large range of different thoughts, there are still clear gaps in the graph that need to be filled.

My ranking algorithm needs work

Version 0.1 implements a simple 'if there are more connections between things then the weight goes up'. Something that I didn't account for is the unfair advantage that is provided to words in longer entries. Simply by being present in a long stream of words, all the other words gain weight in the graph. This is not true for short entries.

Up next

Despite all of this, I still think there's an interesting thread to be pulled at here. I'm going to continue to add to the graph and improve the tools slowly but surely.

Technical Debt Does Exist

This is a comment on an article I read recently

I quite like his comments on the never ending task that is maintaining software, as well as the comment on human nature and the right of passage that is figuring out that ‘doing it all from scratch because you can’t understand old code’ isn’t better most of the time.

But he really seems to misunderstand the point of the word ‘debt’… He gets so close when talks about debt being something that you have to repay but fails to see the connection with repaying yourself for your past shortcuts.

Most of his examples aren’t based on moments where there was a conscious decision of ‘should we do this the way we would want to given enough time or should we bodge it to fit the time constraints, when we know it will make it harder to unpick later?’ amounting to a bodge. These are the real ‘debt’ moments for me – where you are consciously borrowing time from yourself with the knowledge that you will almost certainly have to pay it back with interest later.

Writing it off to ‘maintenance’ is just shirking accountability to your past decisions. There’s a balance to be had but consistently choosing to bodge all the time is just ignoring a large likelihood of a slow down delivery in the long term. Just as always doing it ‘the perfect way’ is short sighted and risks missing the opportune moment to ship/test a new feature.

If you didn’t know you were going to have a problem then I wouldn’t call that debt either.

Status as a Service

thoughts about another article

Assertions throughout the article

Two principles: * People are status-seeking monkeys * People seek out the most efficient path to maximizing social capital

There are no methods for measuring stock and flow of social capital, despite many methods for doing this financial capital.

Growth-stage social startups focus on growth through network effects but struggle to quantify progress on these aims.

The author uses 'Social Capital' and 'status' interchangeably.

The author draws a comparison between ICOs and social networks; tokens are social capital, proof of work earns a token, tokens get scarcer.

Successful social networks make the proof of work mechanism clear upfront.

Some forms of social capital from other status games transfer easily to other networks e.g. media fame and some don't e.g. Tweet quality.

Ammassing social captial in one network is easier when the network is small. Competition is more numerous and more intense in older, larger networks.

Value = scarcity = harder proof of work.

Since people seek an efficient path to gaining social capital, they flock to networks where they find the proof of work mechanism easy (compared to the average person).

Twitter benefitted from an external mechanism to help feedback status, with Favstar and Favrd introducing the element of competition (and feedback on your standing in that competition) externally to the network.

Young people have a bias towards visual mediums over textual ones.

The gradient of your social capital ROI can govern your market share among different demographics. Heavy social media users are hyper aware of different status ROI amongst networks they use.

Graph-based social capital allocation mechanisms can suffer from runaway winner-take-all effects. Combined with a lack of a way to clear out old money, this forms one hypothesis on why old networks die out.

It's not that the existence of old money or old social capital dooms a social network to inevitable stagnation, but a social network should continue to prioritize distribution for the best content, whatever the definition of quality, regardless of the vintage of user producing it. Otherwise a form of social capital inequality sets in, and in the virtual world, where exit costs are much lower than in the real world, new users can easily leave for a new network where their work is more properly rewarded and where status mobility is higher.

Distribution is king.

Replicating a form of proof of work in the same market is likely to fail unless your new thing is better along another axis (utility, entertainment) as well. There's no reason to switch if there are less people on the new network.

It is the unique combination of a feature and a specific graph that is any network’s most critical competitive advantage

What is the one scarcity in the age of abundance? User attention.

Older folks have other forms of social capital available to them, as well as less time to 'prove their work' on any given network. This tends to skew the audience of social networks to the young.

People maintain multiple identities and the ease with which one can cultivate multiple accounts/identities in a network is an important factor for people maintaining context in different social circles.

If you want to know the terminal value of a network, first ask yourself how many people have the skill and interest to compete in that arena.

A feed algorithm (that filters content) can be thought of as a form of interest rate on the currency of user attention. In other words, to purchase the same amount user attention in the future will require a cost (proved work e.g. a post) that changes based upon the algorithm.

People dont' stop using a service because it's too useful. In contrast, social networks can unravel overnight. Status relies on coordinated consensus to define the scarcity that determines its value. Consensus can shift in an instant (a la Memetic Desire).

The fashion industry has taken control of the cycle of devaluation by agreeing and defining trends every season. The tech industry has something to learn in this area as the similarities, in terms of the abundance of rivals, are strong.

Snapchat made a thoughtful move when changing from displaying a single 'Super BFF' relationship (a capped amount of social captial) to an unbounded amount of 'streak' for each contact. This is a clear, direct measure of social capital.

As networks mature, they must swap to focusing more on entertainment or utility in order to retain members of the network. Alternatively, some networks introduce new forms of social capital that can be chased in order to extend their life. For good examples of networks offering something else over time, consider MMO games e.g. WoW, LoL, Fortnite.

FAANGs traditionally struggle with building social networks.

Until we have metrics that distinguish between healthy and unhealthy activity, social network execs largely have to steer by anecdote. Distilling data from 100 – 1000k users dilutes a message massively.

Financial-to-social captial exchanges have no clearer example than those of influencers on YouTube or Instagram.

Almost all successful social networkds are adept at providing both accumulation and storage mechanisms.

If your service is free, the best alternative to capturing the value you create is to own the marketplace where that value is realised and exchanged.

For an individual user, we've standardised on a few basic accumulation mechanisms:

• the profile, acting as the entity through which everything else relates
• followers, generally going upwards steadily
• likes, as a form of continual short-term social captial

For any single user, the stickiness of a social network often correlates strongly with the volume of social capital they've amassed on that network.

Social capital tends to be non-fungible (this somewhat contradicts an earlier point whereby social capital from one network can transfer to another simply by virtue of maintaining the same identity on each – presuming proof of work is just as easily attained in either network), which makes abandoning a network less of an issue.

Requiring that social networks allow people to export their graph and take it to another network might blunt the power of the networks in the social captial axis; forcing them to become more utility or enterainment-driven.

Arbitrage between networks can exist when there are similar proof of work mechanisms. ex. @thefatjewish taking jokes from Twitter and reposting them on Instagram.

IMDb and Wikipedia are two databases that were almost entirely created through the altruistic/status-seeking desires of domain experts.

As we enter a newer age of social networking, claiming ignorance to the downfalls of enormous networks with very little guidance won't be tolerated.

Thoughts

Captial and the use of other, usually financial, constructs can be very useful as a way to illustrate a mechanism. I should explore this more.

Proof of work as a metaphor for social capital value is an excellent analogy.

Feedback about where you are in relation to others is the only mechanism that can drive value creation. Without it there is no way to either: understand where you are in relation to others (removing any notion of status and therefore value) or understand how your actions influence the accumulation of social capital (removing any skill from the proof of work mechanism).

I should look out for arbitrage between new networks as they arise as an intersting thought experiment/predictive exercise.

Digital visual art social network? Proof of work = creating code that is run on the client side and produces the art.

Thoughts on Dukkha

The other day I did some reading around Buddhism and some of the central tenets of the lifestyle. Most importantly, I wanted to understand the underlying principles that shape the rest of the various types of practice. I find that religions and practice rarely hit home 100% for me, however, the adopting the underlying in my own way can be very useful.

As I understand it, 'Dukkha' is the constant dissatisfaction that envelops the human race throughout life. More specifically, it is a dissatisfaction with the constant impermanence of things.

The more that I thought about this observation, the more that it rang true. We are always striving for 'more'. Just as soon as we have achieved a momentous goal, we're left thinking: “What next?”

So pervasive is this outlook on things that we don't even realise that we're looking at things through this lense. In fact, the very first time that I chatted with my partner about how interesting this was she replied words to the effect of “isn't it awful that people struggle with impermanence and end up sad?”

What an excellent insight into the very nature of it we were presented with.

I've been interested recently in ways in which we interact with the tools around us. In most of my life, this boils down to communicating ideas that are inside my brain (in the form of words) to others. In order to communicate these ideas – there are three methods that I can currently call on:

1. spoken word
2. written word
3. written pictures

By far, the method that takes up the largest amount of my time and effort is method one. Because of this, I've been interested in ways that I might improve the brain-to-word throughput that I have. This investment should, hopefully, pay dividends later in life. As is with most tools, the easier and faster that you can get ideas out of your head and into the real world, the more options you have to try out ideas with minimum barrier to entry, iterate and improve.

This brought my search to methods of typing. There are LOADS of keyboard loyouts, all professing to be faster than each other or more efficient. For all of these methods, however, there is a fundamental limitation: they accept that we must type one letter at a time.

I explored a number of new types of keyboard that offer more flexibility, but none of them could overcome the one-letter limitation. However, this morning I read The Art of Chording. It really rang home with my search for a better way and seems like an excellent path forwards.

Stenography, rather than use letters as the building blocks for words, uses sounds. On top of this, we can construct sounds as chords of key presses. What a fantastic system: multiple letters typed from multiple keys at once.

Since Stenography has been around for about 200 years now and is a standard in transcription services, this also offers some confidence that it's working pretty well for real-time conversion of ideas to words.

I plan to practice this daily for the next few months and see how my skills progress/if it actually helps me. To do this, there are a few tools that I'm going to use:

• The Art of Chording
• Plover
• Typey Type
• Plover's 'Learning Stenography' Compendium