mnmlmnl

@mnl@hachyderm.io

TL;DR: I used ChatGPT extensively to edit The missing discussion of LLMs potential. It was cool. I will do it again.

I have been using ChatGPT to help me edit some of my writing, but never for a full blog post. I am not a native English speaker and decided to write (and improve my writing) in 2021. A big part of writing is editing. Editing is notoriously difficult, but it is hard for an author to distance themselves enough from their writing (and thinking) to make the choices that would benefit a reader the most. This is true in music just as it is in writing. While an easy way to get some distance is to let pieces rest, this is not realistic in a professional context, nor do I want to leave drafts of blog posts lying around too long.

An often untold benefit of editing is editing other people's writing and having other people edit your own writing. It is a drastically efficient way of getting a sense of what makes for good writing and what doesn't, and it provides mutual benefit. It is also hard to see one's own words being modified, discarded, and criticized. Learning how to criticize and how to accept criticism is a big part of art education. It should be more prevalent in the education of programmers since code reviews and refactoring is a crucial aspect of programming.

One thing ChatGPT is particularly impressive at is stylistic transfer (rewrite X in the style of the declaration of independence was a viral hit). I figured this would allow me to see my own writing from different perspectives, in a very quick feedback loop.

Editing an article

I usually start my blog posts by freewriting on my freewrite traveler or by collecting some of my threads on [mastodon]. I'll do a first editing pass, playing with the order of arguments, finding additional points to raise in my Zettelkasten, and rewriting sentences to create flow. Lately, I experimented with pushing these second drafts out quickly, which was a good exercise to overcome my perfectionism, although I think I veered too much in the other direction.

I have been using professional editors in the past, and the value they add is very different. They can understand what I am trying to achieve—the point I might be trying to make—and suggest writing a completely different article instead or discarding everything but a paragraph and starting again. These are judgments I wouldn't trust ChatGPT with (although I might decide to use it for ideation; in that case, I would still be operating within the confines of my own limitations).

In this case, I have been working on an extremely long article of using LLMs and stable diffusion for retrieval practice and decided to extract irrelevant rambling into their own blog posts. I asked ChatGPT to suggest different ways of rephrasing individual paragraphs, ultimately settling on the “style of a sober Atlantic article.”

Here's the raw dump of the session, for your perusal. I think it has overall been very beneficial to the wording of the article, although only time will tell. I can definitely see myself becoming bored of its style after using it for a couple of articles. I could see myself deciding that I don't need it anymore except for sentences that give me real trouble. I could see myself building a tool that streamlines the process so that I can use it for everything I write. I could myself pushing it much further than I did here.

For a full comparison of after / before, see this entry in my vault.

Using ChatGPT as an editor, a raw log

Opening paragraph

Original paragraph:

My favorite software tool of 2022 is Github Copilot, which has transformed how I write software. While I studied machine learning in general and neural networks in particular in 2017, I got frustrated by the hype-to-usefulness ratio and decided to move on to other things. Ironically, this was right as the Transformers paper came out. GPT2, GPT3, and then ChatGPT definitely showed me how fast the field could move, and ChatGPT made me realize that a whole new page had been turned.

Prompt for GPT:

Rewrite this paragraph slightly to make it a bit more lyrical, in the style of a new yorker article:

Answer:

As I write this in the waning days of 2021, my thoughts are already with the software tool that has captured my heart and changed my life: Github Copilot. It was back in 2017 when I was deeply immersed in the study of machine learning, and particularly neural networks, that I began to feel a sense of frustration with the disconnect between the hype and the practical applications. I made the decision to move on to other pursuits, little realizing that this was the very moment when the Transformers paper was about to shake the field to its core. The rapid succession of GPT2, GPT3, and ChatGPT that followed only served to underscore the breathtaking pace of progress, and it was ChatGPT that truly opened my eyes to the fact that a new chapter had begun.

Prompt:

Rewrite this paragraphh in the style of a critical the atlantic article:

Answer:

In 2022, it is Github Copilot that stands out as my preferred software tool, fundamentally altering the way I approach writing code. As I delved into the study of machine learning and, specifically, neural networks in 2017, I was increasingly disappointed by the disproportionate emphasis on hype over tangible usefulness. I ultimately decided to turn my attention elsewhere, only to be caught off guard by the release of the Transformers paper. The subsequent emergence of GPT2, GPT3, and ChatGPT served as a harsh reminder of the lightning speed at which the field was advancing, with ChatGPT serving as a particularly poignant signal that a new era had dawned.

Prompt:

Rewrite this paragraph in a more sober, yet personal style:

In 2022, it is Github Copilot that stands out as my preferred software tool, fundamentally altering the way I approach writing code. As I delved into the study of machine learning and, specifically, neural networks in 2017, I was increasingly disappointed by the disproportionate emphasis on hype over tangible usefulness. I ultimately decided to turn my attention elsewhere, only to be caught off guard by the release of the Transformers paper. The subsequent emergence of GPT2, GPT3, and ChatGPT served as a harsh reminder of the lightning speed at which the field was advancing, with ChatGPT serving as a particularly poignant signal that a new era had dawned.

Answer:

As I reflect on my experiences in the world of software development, there is one tool that has had a particularly profound impact on my work: Github Copilot. When I began studying machine learning and neural networks in 2017, I found myself growing disillusioned by the emphasis on buzz over substance. I decided to shift my focus elsewhere, only to be startled by the arrival of the Transformers paper and the subsequent rapid emergence of GPT2, GPT3, and ChatGPT. These developments drove home the point that the field was moving at an extraordinary pace, with ChatGPT serving as a particularly noteworthy marker of a new chapter. Through its influence on my approach to writing code, Github Copilot has been a game-changer for me.

Ultimately, I settled on:

I studied machine learning in general and neural networks in particular in 2017 but got increasingly frustrated by the disconnect between the hype and actual results when put into practice. I decided to move on to other things. Ironically, this was right as the Transformers paper came out. The subsequent emergence of GPT2 and then GPT3 (alongside Codex) was a reminder of how fast the field could move, and ChatGPT made me realize that a new chapter had begun. My favorite software tool of 2022 is Github Copilot, which has fundamentally transformed how I write software.

Second paragraph

Original paragraph:

I don't know if the technology itself is just incrementally improving or if an actual threshold has been crossed. ChatGPT dug itself out of the uncanny valley that was holding me back from using natural language models (while I was immediately comfortable with more pragmatic applications like Copilot, in part due to the care around its prompt engineering ). Shortly after its release, I decided to give ChatGPT a spin as a daily tool for both work and personal research—just as I would a new IDE or programming language; I am still on that journey, and while I feel I haven't really left home base yet, I have already discovered many amazing things.

Prompt:

Rewrite the following paragraph in the style of a sober atlantic article

Answer:

It is difficult to say whether the recent advancements in technology are simply incremental improvements or if we have reached a true turning point. For me, ChatGPT represented a crucial step out of the uncanny valley that had previously hindered my use of natural language models. Despite feeling immediately at ease with more practical tools like Copilot, thanks in part to the attention given to prompt engineering, I was drawn to ChatGPT and decided to incorporate it into my daily work and personal research in the same way I might with a new IDE or programming language. As I continue on this journey, I feel as though I have only just begun to scratch the surface of what is possible, yet I have already made a number of exciting discoveries.

Great sentence:

“It is difficult to say whether the recent advancements in technology are simply incremental improvements or if we have reached a true turning point.”

In general the whole paragraph is better than what I wrote, and I settled on:

It is difficult to say whether the recent technological advancements are simply incremental improvements or if we have reached a true turning point. ChatGPT dug itself out of the uncanny valley that was holding me back from using natural language models. Despite feeling immediately at ease with a more practical tool like Copilot, thanks in part to the attention given to prompt engineering, I was mesmerized (as so many of us were) when ChatGPT came out; I decided to incorporate it into my daily work and personal research in the same way I might with a new IDE or programming language. As I continue on this journey, I feel as though I have only just begun to scratch the surface of what is possible, yet I have already made a number of exciting discoveries.

Creating a subheading

I then decided to ask it for alternative headings, which was less interesting (although, scroll further down for a further attempt).

Prompt:

This is a subtitle in the article I am writing, but I would like it to be shorter and a bit more impactful, while retaining its meaning: Amidst the criticism, I am missing a discussion of their potential

Answer:

“Ignoring the potential amidst the criticism”

Prompt:

no, try again

Answer:

“Overlooking potential amid the criticism”

Prompt:

no, try again, emphasizing that I a missing a discussion of the technology's potential

Answer:

“The missing discussion of potential amid the criticism”

Next paragraph

Original paragraph:

I think LLMs and image-generation networks are powerful tools for learning and helping humans process information. I am always a bit dismayed by the debates around these technologies in my social bubbles, not because the points raised (biased training, generating wrong but plausible-sounding bullshit, non-consensual training on copyrighted data, training and research being mostly in the hands of corporations, cost, and energy concerns, impact on knowledge workers being replaced by machines) aren't important, but because the debate never really focuses on actual benefits to its users, besides the wild overblown promises of the VC/tech/business people.

Prompt:

Rephrase the following in the style of a sober atlantic article

Answer:

As someone who believes that LLMs and image-generation networks have the power to facilitate learning and aid in the processing of information for humans, I am often disheartened by the debates surrounding these technologies in my social circles. It is not that the issues raised – such as biased training, the generation of misleading but credible-sounding misinformation, non-consensual training on copyrighted data, the concentration of training and research in the hands of corporations, cost and energy considerations, and the potential for displacement of knowledge workers by machines – are not important. Rather, it is that these discussions rarely focus on the tangible benefits these technologies can offer to their users, beyond the extravagant and exaggerated claims made by venture capitalists, technology firms, and businesspeople.

I ended up with:

I think LLMs and image-generation networks are powerful tools for learning and helping humans process information. I am often disheartened by the debates surrounding these technologies in my social circles. Not because the issues raised—such as biased training, non-consensual training on copyrighted data, the concentration of training and research in the hands of corporations, cost and energy considerations, and the potential for displacement of knowledge workers by machines—aren't important. Rather, it is that these debates rarely focus on the tangible benefits these technologies offer beyond the exaggerated claims made by venture capitalists, technology firms, and businesspeople.

Asking it for its “reasoning”

I decided to ask it a couple of questions about what it was “thinking.” I have been working with ChatGPT enough to know that it has absolutely no real understanding of what it is doing but instead is just vibing. Yet, these vibes have structure, and vibes are what I am interested in, so it is instructive to ask these kinds of questions at times. It is important to remember that they might sound like some “human reasoning” was going on, but that is just an illusion. The reasoning these models have is ultimately very alien to us.

Prompt:

Why did you do the changes you did?

Answer:

I made the changes to the original paragraph in order to reflect the more sober, measured tone of an Atlantic article. This involved rephrasing some of the more casual language, such as “my social bubbles,” and avoiding words and phrases that might come across as overly emotional or subjective. Additionally, I aimed to add clarity and specificity to certain points, such as by specifying the types of biases and the sources of overblown promises.

Note how the second part is just nonsense if you need a reminder that this stuff is not “intelligent” in any reasonable sense of the word.

Prompt:

which phrases came across as overly emotional or subjective in the original paragraph?

Answer:

Phrases that might be considered overly emotional or subjective in the original paragraph include “always a bit dismayed” and “wild overblown promises.” These phrases convey a sense of strong personal feeling, which might not be appropriate in a more sober, objective style of writing such as that of an Atlantic article.

Prompt:

Are there other changes you made that you think were particularly important?

Answer:

One change that I think was particularly important was the addition of specificity to certain points. For example, the original paragraph mentioned “biased training,” but did not specify the type of bias being referred to. By specifying that the biases in question relate to “training,” I was able to add clarity to this point and make it more concrete for the reader. Similarly, the original paragraph mentioned “wild overblown promises,” but did not specify the source of these promises. By specifying that the promises in question are made by “venture capitalists, technology firms, and businesspeople,” I was able to provide more context and help the reader better understand the issue being addressed.

sic.

Trying to get a subtitle again

Newer LLMs are usually quite good at summarizing text, and it is one of my main uses for writing documentation or writing emails. I know I tend to write too much, and I'll often distill things through ChatGPT, and shorten my writing (rarely actually using ChatGPT's output, because it is very often not exactly what I am trying to express).

Prompt:

What do you think the main point of the paragraph was?

Answer:

The main point of the paragraph seems to be that the debates surrounding LLMs and image-generation networks often do not focus on the actual benefits these technologies can offer to users, but rather on other issues such as biased training and the concentration of training and research in the hands of corporations. The writer expresses frustration with this lack of focus on the potential benefits, and suggests that the extravagant promises made by certain groups may be contributing to this lack of emphasis.

Prompt:

What would be a good subtitle for this paragraph?

Answer:

A good subtitle for this paragraph might be: “Failing to Consider the Potential Benefits of LLMs and Image-Generation Networks.” This title captures the main point of the paragraph, which is that the debates surrounding these technologies often overlook their potential benefits, and conveys the sense of frustration expressed by the writer.

Editing a short paragraph

This shows that the conversational interface to ChatGPT is quite interesting. In many ways, the whole process feels like talking to a well read but utterly stoned friend who is also a pathological liar. I actually have a few real-life friends like that, and I always enjoy spending time with them, because their company is very intellectually stimulating. Having to take everything they say with a huge grain of salt is part of that stimulation.

Prompt:

Can you summarize the following in a single sentence, in the same sober style of an atlantic piece? It seems that most people loudly arguing against LLMs have never tried to work with them. I get it that people are worried about low-quality posts on LinkedIn and flooding of disinformation or plausible-sounding spam, but that phenomenon has already reached a scale that overwhelms the individual.

Answer:

Most individuals who vocally oppose LLMs have not personally worked with them, despite concerns about low-quality posts on LinkedIn and the proliferation of disinformation and plausible-sounding spam that have already reached a level that exceeds the ability of individuals to combat.

Prompt:

Actually, summarize it in two distinct sentences, to separate the two facts

Answer:

Most individuals who vocally oppose LLMs have not personally worked with them. There are concerns about low-quality posts on LinkedIn and the proliferation of disinformation and plausible-sounding spam, but these issues have already reached a scale that exceeds the ability of individuals to combat.

I ended up with:

It seems that most people who vocally oppose LLMs have never personally tried to work with them. Most concerns I see are people worried about low-quality posts on LinkedIn and the proliferation of disinformation or plausible-sounding spam, but these issues have already reached a scale that exceeds the ability of individuals to combat.

More paragraph editing

I lost stamina towards the end, and decided to wrap things up:

Prompt for the final paragraph of the article:

Rewrite the following in a the style of ezra klein on vox.com: If we want to do proper technology criticism, we have to put in the effort to evaluate it fairly and intensively. There are tremendous amounts of interesting aspects to these models when used for personal productivity and great potential for them to be used for self-actualization. The more I play with them, the more I see their limitations, but the more I have scenarios where they turn out to be extremely useful, if not game changers.

Answer:

If we want to engage in truly effective technology criticism, it is essential that we put in the time and effort to assess these tools objectively and thoroughly. There are many fascinating aspects to LLMs and image-generation networks when it comes to personal productivity, and there is great potential for them to be used as aids in self-actualization. The more I experiment with these tools, the more I am able to discern their limitations, but I also keep encountering situations where they prove to be remarkably useful, if not downright transformative.

Prompt:

Do it again, but just for this sentence: If we want to engage in truly effective technology criticism, I think it is essential that we put in the effort to evaluate these tools thoroughly, if only to give solid counterarguments.

Answer:

In order to engage in truly effective technology criticism, it is crucial that we invest the necessary time and effort to evaluate these tools thoroughly, even if only to provide well-supported counterarguments.

Prompt:

Rewrite the following sentence as a punchy closing sentence to the few previous paragraphs: There are many things I find problematic about the current rollout of these models, but that is not going to keep me from studying them. For example, I have my own personal concerns about the misuse of LLMs for disinformation.

Answer:

Despite my reservations about the current deployment of these models, I remain committed to examining them closely – even my own personal concerns about the potential misuse of LLMs for disinformation will not deter me.

Prompt:

How would David Resnick write the following sentence: I find many things problematic about the current rollout of these models, but if anything, that is even more of an incentive to study them.

Answer:

It is precisely because there are so many aspects of the current rollout of these models that I find problematic that I am all the more motivated to study them.

Prompt:

How would Ezra Klein write it?

Answer:

There are many things about the current rollout of these models that give me pause, but rather than dissuade me, those very concerns only strengthen my resolve to delve deeper into their study.

What I ultimately settled on:

If we want to engage in truly effective technology criticism, I think it is essential that we put in the effort to evaluate these tools thoroughly, if only to provide well-supported counterarguments. There are many fascinating aspects to LLMs and image-generation networks when it comes to personal productivity, and there is great potential for them to be used as aids in self-actualization. The more I experiment with these tools, the more I am able to discern their limitations. Still, I also keep encountering situations where they prove to be remarkably useful, if not downright transformative.

I find many things problematic about the current rollout of these models, but if anything, that is even more of an incentive to study them.

Conclusion

I hope this raw overview of what I did for the blog post I just published was instructive. This is all an experiment, and I would love feedback. Do you think the editing was valuable? Do you do something similar? How does this compare to human editing? What are the downsides you see?

I studied machine learning in general and neural networks in particular in 2017 but got increasingly frustrated by the disconnect between the hype and actual results when put into practice. I decided to move on to other things. Ironically, this was right as the Transformers paper came out. The subsequent emergence of GPT2 and then GPT3 (alongside Codex) was a reminder of how fast the field could move, and ChatGPT made me realize that a new chapter had begun. My favorite software tool of 2022 is Github Copilot, which has fundamentally transformed how I write software.

It is difficult to say whether the recent technological advancements are simply incremental improvements or if we have reached a true turning point. ChatGPT dug itself out of the uncanny valley that was holding me back from using natural language models. Despite feeling immediately at ease with a more practical tool like Copilot, thanks in part to the attention given to prompt engineering, I was mesmerized (as so many of us were) when ChatGPT came out; I decided to incorporate it into my daily work and personal research in the same way I might with a new IDE or programming language. As I continue on this journey, I feel as though I have only just begun to scratch the surface of what is possible, yet I have already made a number of exciting discoveries.

I think LLMs and image-generation networks are powerful tools for learning and helping humans process information. I am often disheartened by the debates surrounding these technologies in my social circles. Not because the issues raised—such as biased training, non-consensual training on copyrighted data, the concentration of training and research in the hands of corporations, cost and energy considerations, and the potential for displacement of knowledge workers by machines—aren't important. Rather, it is that these debates rarely focus on the tangible benefits these technologies offer beyond the exaggerated claims made by venture capitalists, technology firms, and businesspeople.

It seems that most people who vocally oppose LLMs have never personally tried to work with them. Most concerns I see are people worried about low-quality posts on LinkedIn and the proliferation of disinformation or plausible-sounding spam, but these issues have already reached a scale that exceeds the ability of individuals to combat.

If we want to engage in truly effective technology criticism, I think it is essential that we put in the effort to evaluate these tools thoroughly, if only to provide well-supported counterarguments. There are many fascinating aspects to LLMs and image-generation networks when it comes to personal productivity, and there is great potential for them to be used as aids in self-actualization. The more I experiment with these tools, the more I am able to discern their limitations. Still, I also keep encountering situations where they prove to be remarkably useful, if not downright transformative.

I find many things problematic about the current rollout of these models, but if anything, that is even more of an incentive to study them.

(Disclaimer: I used ChatGPT heavily to edit the final form of this article, and I documented how in a follow-up post)

TL;DR: it doesn't matter how many tabs you have or how you close them; the value I get from them comes down to intellectually engaging with them fully, which is exhausting and rewarding. The most valuable practice I found is “generating” out of tabs, and for that, sometimes, just the tab title is enough. No tool is going to save you.

I have a few modes of “consuming” these tab piles.

In-depth studying

In-depth studying can be, for example, taking one day to go through one page of a math book. It's sitting down with a tutorial and actually going through it, doing all the side exercises, and then reflecting upon it. It's slow af, but it's gratifying, and of course, I learn things. Over the long term, the value of that learning diminishes, sometimes very rapidly, depending on what I focused on. I learned awk and R repeatedly, at times over months, and it's all gone. What stays are some more profound insights that were uncovered just through sheer focus. This is the “it takes a full day to close a single tab” mode.

Of course, a tab could be a textbook that actually would take three semesters to work through, so there's a wide range in what “in-depth” itself means.

Reading and annotating

Reading and annotating is where I sit down with an article (for example, using Reader) and read it to really engage with it. I don't just highlight interesting passages; I put myself in the mindset of having a conversation with the author, of pitting my ideas against theirs. This is a pretty high-intensity activity, and when I do this over the weekend, it takes me about an hour to process a single tab.

This way of processing information is what I find has the most “return on time reading.” I have a reasonably elaborate Zettelkasten system going, and filing thoughts that come out of articles, along with notes, is very productive; it often leads to blog posts. Over time, I have found how to make highlights and quotes, and crosslinking work for me (it wasn't easy!).

The downside is that usually, for every tab closed, 80 more get opened. I can reasonably process about 5-6 tabs this way during a work week, maybe ten if I'm pushing it. On holidays I would average 5-6 per day because I get more efficient once I get into a particular “scholarly” routine.

Reading for entertainment

This would be just reading a tab for fun. I only do this if I just opened a tab. I rarely go back to an old tab and then just read it for fun, as it's usually just more dopamine-rewarding to open a new tab on HN or scroll Mastodon. This is reasonably fast and generally pretty transient in terms of “return on investment.” Sure maybe over the years, I get something out of it, but I consider it entertainment (which is great!).

I think there is a lot of value in just looking at the title of a tab, quickly scrolling through it, and either discarding it or writing a small paragraph about it. Writing a short entry about why I think the link is interesting, what is interesting about it, how I found it, and what other pieces of knowledge relate to it is the most valuable thing to capture. It puts the rich context I have in my head, at this moment, onto paper.

If that little paragraph is stored in a suitable location, it means that the next time I want to study that topic or look something up, I will find it, along with its link, and immediately get context. That is actually extremely valuable. This filing of links is something I am not very good at, and I definitely want to work on making that part of my workflow better.

It's very easy to think that I will remember the nuances of my current thinking in the future, but that's a delusion. I need to write it down. Otherwise, it's gone.

This not only creates a lasting way for me to find that link again (this is where a crosslinked system becomes very useful because it allows me to crosslink the text I am writing at the speed of typing, really)—it seizes the opportunity to do some generative thinking, creating new knowledge that is entirely mine.

Just the fact of writing that little paragraph explaining why I want to read the link in the future probably gives me more “value” (as in, it will help me generate my knowledge in the future) than actually reading for leisure (see 3), because I actually “created” something myself.

This takes about 2-5 minutes per tab (sometimes faster when groups of tabs relate to the same concept). It is also exhausting work; if I do this for two hours, I'll be ready to plop down in front of Netflix and purge my brain with reality TV.

A concrete example: filing the Fennel homepage

A concrete example: I stumble across the the Fennel programming language. Incredibly interesting to me, but also something I feel would deserve a few months, if not a year, of attention to “really” get it. I can file it away under Lisp / Lua / Programming Languages and my daily log in my obsidian vault, maybe skim the website and make a bullet point list of points I find interesting, and link a HN discussion.

Just for fun, I created the entry for Fennel Lisp by “processing” the index page to show what that is like: https://publish.obsidian.md/manuel/Wiki/Programming/Fennel+Lisp

I mentioned Fennel Lisp in June in my daily notes, so I also linked that. If you step through the vault, you can see how I slowly moved from:

  • pile of links, to
  • pile of links with a single sentence per link, to
  • full paragraph with a lot more thinking about how I relate to the link.

Which is better?

So, is any of these ways of processing open tabs better? I like all of them, and I definitely had to build workflows for 1, 2, 4. I am content now knowing that there is no solution, and feeling like you can process 800 links a day is impossible. Instead I focus on time-boxing “quality time”, and just close all the tabs once I'm done, there'll be plenty of high-value quality time the next day.

We know that even slight changes in the user experience or algorithms of social media software can have snowball effects and drastically change how people interact.

But, no-one is omniscient. Even the best intentioned developers, the most carefully planned and discussed feature can, in hindsight, turn out to have undesired effects.

Not only should we design our social media experiences to foster an individual's agency (individual opt-in, instance opt-in, facilitate instance migration), we should also design features and processes such that we can reconsider our decisions and potentially roll them back easily.

On a software level, this could for example mean:

  • separate feature releases from bugfix releases
  • use a plugin architecture, or at least enable entire features to be toggle-able at instance level
  • structure open-source projects such that forks can benefit from each other's work without excessive fragmentation
  • always think of frontend/client developers first, because they ultimately shape what the experience is for most users
  • have features degrade gracefully for people who opt-out

On a “governance” level:

  • clear guidelines for the adoption of new features (could be by vote, by decree, by dice-roll, as long as instance members agree to it)
  • processes to check back in and see if people still think the feature is a good idea
  • this could include beta programs or A/B testing, as long as users consent to participating
  • processes to roll back previously agreed upon features

On a community level:

  • understand that people want different things out of social media—pretty much everybody taking part in these discussions is of good faith and deserves respect
  • de-federation is the mechanism that enables us to have productive discussions around these topics. If, after listening to each other, and understanding each other's incentives, we decide we are fundamentally incompatible, we can sever all links yet both still contribute to the fediverse itself

Disclaimer: I say we because I understand I am not operating in a vacuum, but these are just concepts I am working out for myself. I welcome feedback!

It is easy to underestimate how much work it is to keep a “usable” Obsidian vault. While the effort varies, a “properly” maintained vault (as in, I am comfortable with its level of messiness, and I know I will find the things I need to find without unreasonable friction) takes me about 10 hours of work per week.

The fact that I refer to it as an Obsidian vault is misleading. It is not about the tool—I find my use of Obsidian to have no friction anymore. I use it without any plugins and just write and write and write simple cross-linked markdown files.

What takes effort is taking your intellectual output and shaping it into something that has lasting value.

It's something I find missing from most PKM tooling/productivity workflow discussions.

Establishing context

There is, of course, something about being comfortable with your tools. But what is really hard and time-consuming is understanding how things will get used in the future.

Capturing is one part, but perfect capture will not lead to useful knowledge. You need to synthesize all the links, citations, highlights, thoughts, diaries, atomic notes, drafts, and ramblings you wrote and stuffed into your vault and put them in context.

Uncovering context is an activity where experience plays a big role. It means understanding the landscape of your thoughts and the landscape of your output. You need to know how you think and how you produce. This is not something you can just look up or copy. No one thinks like you, and the best you can gather from all the methodologies and productivity advice is the inspiration for building context and what other people find useful.

Anybody trying to convince you there is a proper way is dead wrong. It also doesn't mean there isn't value in understanding what they are doing and even adopting it.

The shape and function of knowledge

How you think helps you figure out how to reformulate things. It creates knowledge that has shape. When knowledge has the right shape, it will assemble with other knowledge with the right shape.

You can't just take someone's thoughts and make them with your own.

But your knowledge also needs to have function. This is, of course, related to its shape, but it is informed by what you will do with the knowledge.

How I write and store things I want to use for study is different from how I store things I need to write work documents is different from how I store things I need to create new thoughts is different from how I store knowledge I want to use for writing.

One big shape factor is how to “cluster” notes. This is where most of the debate is, and again, I think it is deeply related to your mode of thinking and production. Tags / folders / index notes / dynamic views / lists. None of this matters if you don't know the shape of your thinking and how knowledge and smaller thoughts relate to bigger nexuses.

Once you know the shape and function of your clusters, it is just a matter of encoding them with the tools at hand.

How I create shape and function

I'm currently in a very generative phase—I have been writing a lot of notes into my analog sketchbook, I have collected hundreds of links and highlights in Reader, I have written countless threads on mastodon, short articles on write.as.

However, none of that knowledge has found its way into my vault in a way that will make it useful in the future. What I need to do to make it useful is: – create Zettelkasten entries for atomic thoughts that I know can be crosslinked – structure notes to establish clusters of thoughts. This is maybe the hardest work. It takes a while and a lot of writing to start discerning these clusters – develop a new methodology for dealing with the large number of highlights and quotes I am gathering from Readwise (it is a new tool).

One of my interests lately is the fediverse, mastodon, activitypub, social media. After 5-6 weeks of research, thinking, writing, I am finally starting to understand what the skeleton of my Obsidian vault is going to be to leverage my work.

For me, it is both a purely “mechanical” workflow. I file scraps into: – wiki entries (pure knowledge) – quotes (cleaned up quote documents I can quickly search and drop into an article) – zettelkasten notes (clearly formulated, punchy thoughts) I also try to make my rambling (diary, mastodon threads, freewriting, notes about articles) reasonably discoverable (usually by cutting out blog-shaped parts of them and calling them “drafts” and referencing them in the relevant ZK and wiki entries).

But it is also establishing what I call “structure notes,” which you can think of as “book-sized” clusters. They are concepts that group many ideas, and I can probably write a medium-sized book about them.

For the fediverse thinking I have been doing, these initial structure notes are starting to gain shape. They are shorthand for entire arguments I have been having with myself. The title of these clusters encodes much more context for me than can be shared with others. Think of them as package names in a codebase, for example.

  • social structure of the fediverse
  • convivial technology in the fediverse
  • moderation tools in the fediverse
  • product thinking and the fediverse
  • technical resources and projects ideas about the fediverse

Conclusion

This is a quick write-up because I am just starting to go through all the content I have generated in the past few weeks, and I forgot about the time required to do a good job. It is also nice to see how there are phases to all this. I can stop gardening my vault for a couple of months and return to it with no problem.

Every couple of months, some part of my knowledge management workflow (most of which I do in my Obsidian vault ) crosses a threshold. Something has grown to the point of not being manageable anymore; a new tool has proven to be really useful and its use has to become more codified; something has turned out to not work as expected and needs to be readjusted or removed.

I've been using Readwise's Reader for a month now (since it came out of closed beta), especially extensively in the last week. It made me pass one of those thresholds.

Readwise Reader

For those who don't know Readwise, it is a website that collects your highlights from many different sources (ebooks, articles, webpages, ...). It combines this highlight management functionality with a spaced repetition mode where it will replay a bunch of highlights to you every day. I've been doing it, and it's been fun, but I don't know how much value there is there for me (I've got bigger plans for spaced repetition in 2023).

Their Reader application is a central hub for all this highlighting. No need to annotate things in kindle and then import it to Readwise and then transfer that to Obsidian. I can now import PDFs, ebooks, webpages, and random snippets of text right there into a centralized hub. I can then highlight, annotate, tag these documents and import the resulting highlights into obsidian. The process is quite smooth!

Dealing with internet writings

I am an avid consumer of internet content, usually coming from a small variety of sources: – links I find on hacker news, lobsters, RSS feeds... – links I find on social media – links I find while deep-diving into one topic

Often, these links are interesting to me but not relevant to what I am currently working on. I used to file them in my Evernote vault and never look at them again, and in late summer shifted to filing them into todoist so that I could go back through them properly. This “file and forget” action is very healthy, as it allows me to feel OK with moving on, despite my curiosity.

As my note-taking evolved, I realized the value of processing these resources and making the nuggets of information they contain discoverable. This is a lot of work, it requires you to read the article, and take notes extracting “reusable value” that can be crosslinked into the vault. It is also work that I find extremely pleasurable and inspiring, too inspiring in fact...

The problem with extracting value

What I would do is go to a coffee shop on the weekends, open my Todoist inbox, go through the links and read and process articles while writing them up, either on paper index cards or in my Obsidian vault. Creating reusable value out of them for me involves: – creating Wiki entries with “hard facts” – creating Zettelkasten entries with “atomic ideas” – writing about it (blog posts, book chapters, even just diary entries)

There are two problems with this approach: – the friction to getting into “information processing” mode is quite high. I need to get into work mode, I need to have 2 tools open, and I need to focus. But I do read a lot of content on half-flame, for relaxation. This means that I only do this on the weekends, in reasonably exhausting multi-hour sessions – I get way too excited. Once I start digesting articles and coming up with Zettelkasten entries, and writing things on index cards or in my vault, hundreds of ideas come to me, and I will end up processing about 10 articles but added about 40 more to the list while doing so.

Getting way too excited doing knowledge work is a thing I love about my brain: I get a lot of value out of these sessions. I've been milking 3 sessions from August writing down notes about “prototyping in software.” But it really doesn't help boil down the knowledge I've encountered. I also end up bouncing the same 3 concepts around for way too long, when more perspectives on the subject would be more helpful.

Do I even want to “boil down articles?”

Now, why do I even want to process all these links? Am I not happy just processing 10, getting a lot of value out of them, and moving on?

One thing I find lacking in my knowledge management currently is the lack of quotes and references to other sources. In my current way of approaching external articles (taking notes, coming up with my own ideas, making Zettelkasten cards ), I have more of a “discussion” with the document than an “extraction” process.

The downside is that often, the actual meaning of the document is not preserved, and I just write down my own biased ideas. And the other is that I don't have good quotes, but good quotes are actually great in blog posts.

What I want is a way to expand my set of ideas, by clearly identifying where they came from, who said them, and in which context, and only then start adding my own notes.

Highlighting and extracting value

I didn't use to highlight things in books and articles, because I would have to put much effort into transporting them into my vault. This changes quite significantly with Readwise and its Obsidian import plugin, as well as the Reader application.

It's been nice to have a place with a queue of articles and know that at least the fact I read them is going to be preserved (and even synced into my vault). Because of the friction of my current process (link into todoist, read carefully and process on the weekend), I would rarely ever productively consume content during the week. Either I would read an article but not take notes and whatever I got from it would mostly be lost, or I would extract “too much” value and not increase the breadth of my knowledge.

With Reader I feel I can just mindlessly scroll through the stuff I saved, realize that I don't have that much interest in it after all, and still preserve the one or other nice sentence in there that I can later in Obsidian cross reference. I will often get through 20-30 articles per day that way!

Where to go from here?

I started devising a workflow to go through the imported snippets in Obsidian (since there are now more than 100 in there), and finding ways to crosslink and file them as they come in. This in a way is the “knowledge work” part of what I used to do on weekends, where ideas are extracted, and things are put in context.

Because it has now been broken off from the “consumption and highlight” part, it feels much more approachable, and also something that can be done on the side, because I only need to recall the snippets for the topic I am working on right now, and can immediately crosslink digested nuggets, instead of having to read original sources.

Too many books this year, but these stood out:

  • the joy of abstraction by eugenia cheng, where oddly enough I've barely read 7 chapters and it's given me so much.
  • write useful books by brad fitzpatrick. eminently practical, super short.
  • seven sketches in compositionality, I've faked my way through 5 chapters, it is really fun and not too hard for me to get into.
  • the secrets of consulting by gerald weinberg – hilarious and wise, it really helped me reframe what I'm actually providing as I get into advisory consulting.
  • unmasking autism by devon price – amazing, deep and practical book about what it means to be autistic and how to overcome the trauma of living in a world where you don't fit.
  • intelligent embedded systems by Louis Odette – this is a book you have to program, which I didn't, but it's absolutely wild. build a VM, then a forth, then a lisp, then a prolog, then an expert system, all for concrete embedded applications. all with source code in 300 pages.
  • augmented lean by Natan Lindner (disclaimer, he's a friend and I consult for his company) about what digital technologies can bring to make manufacturing and supply chains more human and empower workers to have agency in an increasingly automated and monitored world.
  • old but gold: SICP JS version, which gave a new perspective on a book that fundamentally shaped my life. I've written more about it at dev.to.
  • hacking capitalism by kris nova – you really have to chew through it to extract what she wants to get at, and it could have used more editing, this has helped me understand a lot of things about operating in a capitalist world as a technologist. She also is building an amazing community around hachyderm.io which i recommend looking into.
  • weinberg on writing by Gerald Weinberg – a zettelkasten approach to writing, which fits my brain very well, and explains the deep wisdom, erudition, and quirkiness of Weinberg's writing.
  • old but gold: patterns of software by Richard Gabriel, which has to be one of my favorite books on software writing, and which I hadn't read in a decade
  • old but gold: the plenitude by rich gold, which is just such an inspiring book about an artist turned technologist behind a lot of the fun stuff at Xerox PARC
  • less life-changing but fun to work through: cloud native go about building “modern” software with go, with a lot of concrete takes and examples.

I really enjoy writing SQL builders. By that I mean that instead of writing a SQL query, I write a little struct along with a builder pattern that allows me to build up the query. This gives me a proper API documenting the query's intent, as well as a way to do schema migrations in the future while keeping the power of raw SQL.

This is three times more so these days, where most of this code can be TAB-completed with GitHub copilot. I wrote the following example in about 2 minutes.

type UserQuery struct {
	filterEnabled bool
	filterValue   string
	getFullInfo   bool
}

func NewUserQuery() *UserQuery {
	return &UserQuery{
		filterEnabled: false,
		filterValue:   "",
		getFullInfo:   false,
	}
}

func (uq *UserQuery) Filter(filterValue string) *UserQuery {
	uq.filterEnabled = true
	uq.filterValue = filterValue
	return uq
}

func (uq *UserQuery) FullInfo() *UserQuery {
	uq.getFullInfo = true
	return uq
}

func (uq *UserQuery) Build() (string, []interface{}) {

	binds := make([]interface{}, 0)
	selects := []string{"id", "name", "email"}
	wheres := []string{"1=1"}

	if uq.filterEnabled {
		wheres = append(wheres, "name LIKE ?")
		binds = append(binds, uq.filterValue)
	}
	if uq.getFullInfo {
		selects = append(selects, "phone", "address")
	}

	return fmt.Sprintf(
		"SELECT %s FROM users WHERE %s",
		strings.Join(selects, ", "),
		strings.Join(wheres, " AND ")), binds
}

func main() {
	q := NewUserQuery().Filter("Manuel%").FullInfo()
	query, binds := q.Build()
	db.Query(query, binds...)
}

One thing I love doing with this pattern is keeping performance metrics and logs of executed queries, which is easily done and configurable because I can add all kinds of things to the UserQuery class itself.

I've been an avid user of GitHub Copilot since it came out, and it has transformed how I write programs. I now am able to spend quite a bit of time on nice error messages, covering all edge cases with unit tests, devising “complete” APIs, and doing proper error handling.

I use Copilot for basically two use cases: – generate code I have already written in my head – fish for API functions I know exist but can't remember the name of. This often leads to discovering patterns I didn't yet know of

I do tend to turn copilot off when I have “real” programming to do because it tends to generate nonsense if it doesn't already have 2 or 3 base cases to latch onto.

I do sometimes use it as an idea generator, writing a little line of text and waiting for it to suggest some function names or structures. I've found good ideas that way, for example, suggested serialization structures that had more fields than I would have thought of adding, or completing accessor functions that I didn't fully think through.

ChatGPT is on another level

ChatGPT is on another level. I had planned to use it for the day to see what I could get out of it, and I have used it to generate the following: – a list of concrete instances of structures that test edge cases of an import function – a set of terraform resources to deploy a cronjob on AWS Fargate – a makefile to build and push a docker image to an AWS registry – a PHP script to collect and zip a set of resource files, naming the zip file according to the timestamp and git hash and branch, and pushing it to an s3 bucket – a set of next.js endpoints to implement an ecommerce cart functionality

None of this is code that is in any way complicated to write, and I had to redirect GPT from time to time, but the fact that I was able to build all of the above in about an hour's time is absolutely mindboggling. Not only did it generate reasonable answers, but it did a stellar job of documenting what it was doing.

It means I can fully focus on intent, testing, documentation, developer UX, unit testing, and the “complex” part of my job, without having to hunt down terraform documentation or copy-pasting some ECR Makefile magic.

The discourse around ChatGPT and Copilot is weird

I think most of the discussion around ChatGPT and Copilot is disconnected from the experience of actually using it. I'm talking about: – the danger of them producing broken code, – that you can induce it to regurgitate complete functions verbatim, – that it will make us stupid and careless

Both ChatGPT and Copilot confidently generate a lot of wrong or nonsensical suggestions. So does my IDE auto-completion. I found that anytime I dare Copilot to generate something I don't already know how to write, or something I have already written but with different arguments, it will generate something sensible on the surface but wrong. The effort it takes me to code review Copilot's suggestions is mentally more taxing than writing it myself, and after a while, I started to “feel” what Copilot will do well and what it won't.

Will people get bitten by blindly accepting Copilot's or ChatGPT's suggestions? Sure. If you care about your application, you will quickly notice that things are not working. Poor programmers can write broken code entirely without machine help. Good programmers will quickly realize that even the best code, machine-generated or not, will need review, testing, and validation.

Solving problems is about more than coding a solution

More importantly, you need to already have a good understanding of the solution to your actual problem to get a usable output.

Prompts that ChatGPT and to some extent Copilot do well on, like: – I need a next.js handler that takes a JSON structure X and stores it in the table Y in MySQL – Write a makefile to build a docker image that runs mysqldump on X and uploads the result on Y

require a level of planning and understanding of software architecture that requires “strategical” thinking. These smaller tasks are in themselves extremely abstract, only peripherally related to the real-world problem of “selling plants online” or even “backing up our production database.” These tools are made to work at that level.

If I were to ask ChatGPT “please backup my database,” why would I expect its answer to be any better than one of the hundreds of competent SaaS offerings out there? For its answer to be good, I need to guide it so that its answer fits well into my concrete codebase, team structure, operations, and planning. This is hard work, it requires thinking, communication, prototyping, learning new technologies, and knowing the business, the codebase, project constraints, requirements, and team topologies.

That is exactly what I enjoy doing as a principal engineer: high-level technical strategy, with very sharp decision-making when it comes to concrete problems while giving team members the opportunity to own and shape the result, or a more menial AI the pleasure to fill in the blanks.

I have been dabbling with category theory for a while, but I'm actually still unsure about really “studying maths” and how serious I want to be about it.

I don't deal much with applied mathematics. I don't need algebra for geometry, calculus for modeling, or advanced statistics for data analysis. I want better abstractions for refactoring legacy code and robust systems programming.

I'm interested in the insights abstract mathematics gives me regarding my programming. Yet, I understand that to fully realize those insights, I need to study mathematics the way mathematics wants to be studied: I need to be more careful and intentional about it.

Working through books

I now have a fair amount of “proper” mathematics books to study, but only a few hours per week that I want to dedicate to it (I have an open-source side-project I'm excited about, and a book and a blog to write)..

The books are: – Aluffi's Algebra Chapter 0 – the more popularized but still rigorous Seven Sketches in Compositionality – Awodey's Category Theory

These along with more “informal” material like: – Category for programmers by Bartosz Milewski, both the book and the lectures – Programming with Categories, the online lecture by Fong, Spivak and Milewski – Various functional programming and Haskell meetup recordings

I want to a least follow the proofs in the first set of books—maybe do some proof sketches. I feel that actually doing proofs only maintains interest enough if I can cross-check them with other people. Studying rigorously needs community. I'm not really a puzzle guy—I'm way too impatient.

One way to potentially make proof work more relevant (fun!) for me might be to look into theorem provers. I am afraid of the steep learning curve of doing computer proofs in fields that I am not acquainted with. I had reasonable success going through programming language theory coq tutorials; yet I would often hit a wall where I was unsure if I understood the theory, and was just struggling with the theorem prover, or if I had the theory wrong (even though I was familiar with the field, both on a theoretical and practical level).

Keeping track of my progress and insights in blogs

A fruitful way to solidify my progress, as messy as it might be, is going to be to write about my insights after each study session in this blog. I will be often wrong, and often messy—basically, I will publicly show that I have no clue about what I'm talking about. This always holds me back on my main blog, because I know it gets a fair amount of scrutiny and I am quite sensitive to criticism. I only want to talk about stuff I can back up confidently (sadly I only feel confident about a very few things, mainly that I am thinking my own thoughts.)

In these blog posts, it will become apparent I barely know what a set is, and have to go back to definitions every couple of minutes. One thing I realized however is that this lack of rigor doesn't necessarily make the insights less valuable. After all, I'm not a mathematician, I'm a programmer, and overall my software seems to work well enough. So what if I use fancy words wrong to actually get the work done? The best I can do is work at using them correctly more often.

Extracting value out of theoretical texts

I find that I get the most important ideas out of the very first lines of the introductory chapters. The motivation for a field of study is often what needs to stick because the theory will be forgotten soon enough; working through the theory in detail is however a good way to make the motivation stick. A Zettelkasten is a hack to make the insight retrievable without having to internalize it.

Mathematical texts are often devoid of motivation that I can relate to. I am starting to understand how to take abstract structures and start mining my knowledge (design patterns, applications, domains, programming languages theory) to see if I can something to relate it to. Structures like monoids can be easily found where they obviously relate to mathematical operations, but I am now trying to think out of the box. Can a UI be a monoid? Is there a monoid in handling database connection failures? Do errors form a category? Often, these bring up nothing—even then, I know I need to come back to them once I learn more theory.

For example, I am trying to understand what limits and colimits are about. What are they trying to tell me, why do people study them, and what use do they derive out of it? Can I come up with my own?

Seven Sketches Chapter 1 – Generative Effects

A valuable insight (which I capture in Zettelkasten notes) comes right at the beginning. They talk about generative effects. If I understand it correctly, if we have a system with a compositional structure (for example, virus transmission), it is possible to observe it partially (the observation doesn't conserve the operation of transmission) and end up with seemingly surprising results.

For example, we might observe that individual A had contact with individual B in a certain county and that individual B had contact with individual C in another. In each county, we didn't observe A having had direct or indirect contact with C. But putting both county observations, we clearly see that A had contact with B had contact with C.

This is something that comes up over and over again, for example, partial logs, in my day-to-day job. It applies literally everywhere, it's nothing special per se, but I now know that this has been studied in detail at a very high level of abstraction. This will allow me to group any “partial observation” pattern I can infer from my work into the same mental box.

Now, how is this related to the rest of the chapter? Ordering, partitions of sets, equivalence relations, monotone maps, meets, and joins? I will have to work that out. It will I'm sure shed light on parts of logging and observability I often struggle with: what IDs are assigned to what log, how and where do I store individual entries, what do I do with structured data and how do I index it, how do I fill in or indicate missing data, and how do I ultimately process reports. Ad-hoc approaches often work, but they take effort, and I never feel confident about having covered my bases.

I used to have the same issues with things like concurrency, but by getting more closely acquainted with monads and monoids, I now feel that that is a “solved” problem for me.

Another area I feel better knowledge of ordered sets and monotone maps will help is the startup and termination of concurrent processes. Feel free to call me out in a couple of weeks if I don't circle back on this.