🌿  back

21.48 YRS

conceptual foraging intuitions

Velma’s write-up got extended with a few qualitative samples which show what kinds of beliefs can be inferred from what kinds of tweets using the three approaches we explored. Also, a paper based on oneironomicon got accepted at a new conference on human-AI interaction!


This post is an update of sorts, covering what I think is a promising strategy in uncovering pockets of new ideas. By new ideas, I mean instances of H-creativity (i.e. historical creativity), rather than P-creativity (i.e. psychological creativity). While P-creativity refers to any discovery which is new for you, H-creativity is a tiny subset which only contains ideas that are new for all of humanity.

A disclaimer is warranted whenever we talk at this meta level. A while ago, I thought it was reasonable to work solely at this meta-science level, making the study of ideas your object of study. I don’t fully buy that anymore, as I think it’s quite useful and reassuring to ground your work in something a bit more concrete. Having both the supply and demand of tools for thought collocated in the same mind accelerates both their creation and use. For me, my more concrete interests are along the lines of:

The fact that any single one of those could busy one for a few lifetimes is not the point. The point is that they’re all more concrete endeavors than ideas about ideas. And again, I’m now of the opinion that both concrete and meta interests should coexist for best results.

With this disclaimer in mind, let’s move onto modeling the problem in a bit more depth. Imagine a space of thoughts as a backdrop, roughly corresponding to functional space in last week’s article. This space is populated with valuable gems here and there, scattered across the space, which represent ideas. The situation is hardly homogeneous: there are regions of this space which have a higher density of conceptual gems than others. People want to find those gems in order to trade them in, or just as a hobby.

Individuals can only look for gems in one place at a time, but they have a peculiar property. Their focus locally jumps around uncontrollably from moment to moment, but they have control over which broader regions they spend time fidgeting around. For instance, Paul Christiano might decide to spend a lot of time in the region of AI safety via debate, but he’ll have less control over when specific ideas will pop up. He’ll extensively bounce around that region of idea space, as he believes it’s particularly promising, but it’s difficult to predict when he’ll run into particular insights.

I often use graphs and spaces as formalisms, but here there’s a better tool for the job: probability distributions. Each person interested in mining such conceptual gems has a unique model of which regions have a higher density of them. One could then spend time bouncing around regions believed to have a high density of conceptual gems, hoping to run into them. However, it’s impossible to visit all locations, the space is just too big for a human lifetime. A function which maps a region of space to such a density is called a probability density function, or PDF. If you want to visualize a PDF over a 2D space, it’ll look a bit like hills: tall hills will be present at 2D regions thought to be particularly dense, while valleys will be present at 2D coordinates deemed less interesting. If you want to visualize a PDF over a 3D space, you’ll have to make do with a heat map cutting through mineral ores.

Probability distribution over 2D space. Altitude and color both indicate density, with the redundancy aiming to get the point home. (Source)

Okay, we’re getting there. Now we can talk of Paul Christiano having a mental PDF which perhaps peaks at AI safety via debate, perhaps has a few modest hills around other approaches to the problem like transparency tools, and approaches flatness in most other regions of idea space. He’ll then rejection sample his way around this landscape, spending time at high altitude.

Another PDF which deserves particular mention is the aggregate focus of all conceptual foragers. Let’s call this the aggregate PDF, as opposed to a personal PDF. The entire scientific enterprise spends more time on some topics than others. By definition, it spends more time on conventional topics and approaches, and less time on unconventional ones. Here’s the first practical insight since the beginning of this article: relative neglect by the entire scientific enterprise influences the true value of a region of idea space. If you happen to run into even one or two hefty gems in a region widely thought to be worthless, then those are particularly valuable. It’s likely that there’s an unexplored conceptual ore waiting to be mined. You might want to intentionally adjust your personal PDF to account for it, which takes effort. More on this intentional tweak later.

Here are some related practical insights. You might want to be intimately familiar with the aggregate PDF so that you can better gauge the true value of a region of idea space based on neglect. However, this is both a blessing and a curse, because being too familiar with the conventional approaches adopted by the aggregate PDF influences your own personal PDF. You become attuned to what’s deemed to be sensible inquiry™ at the moment, which inevitably changes over time. However, you might be able to get the best of both worlds if you access an external model of the aggregate PDF, without needing to hold it in your head. You might search for an idea and count the number of search results (e.g. websites, papers) as an estimate of the aggregate PDF in that region of idea space. Alternatively, you could just ask an expert or poll a community.

The aggregate PDF also takes a while to shift and update its focus in light of promising new approaches. There are tenured careers on the line who’ve invested time into certain regions and are in charge of staff at the bottom of the food chain to further investigate those. This means that if you’ve identified a promising and neglected region, you have a first-mover advantage before the mammoth catches up (assuming there’s really something there). This is related to how microverses of knowledge in the conceptarium connect both past and future ideas across users. The sharing moment is quite arbitrary, and it’s the region as a whole that matters, which gets mined over time.

Okay, say you found a couple of gems in a really unexpected region. This sparks your interest and nudges you to look into it more. In reality, I’ve found that simply wishing this to happen or making a mental note is not that effective. You’re constantly bombarded with the status quo of the aggregate PDF, and it takes active effort to be optimistic in the face of uncertainty. I found that there are a few steps which help temporarily shift the PDF towards a promising region. You can think of these as means of granting a region of idea space a temporary visa for “sensible-land,” with possibility of extension.

I’ll use as an example an idea I’ve entertained over the past week or so, though its actual contents aren’t essential to the meta level we’re focusing on in this article. The idea is that you could tweak the dynamics of an AI (e.g. the way GPT-3 moves towards talking about certain things and away from others) by defining pairs of attractors (e.g. P = “Humans are amazing.”) and repellers (e.g. ¬P = “Human aren’t an efficient arrangement of atoms in my pursuit of paperclips.”). A set of such preferences could be enforced by recognizing promising behavior via velma-like tools. I thought it was unlikely because it connected alignment and magnetism (see below), a mix I never heard of, hence assumed to be neglected via an availability heuristic. Back to the meta level: in order to boost the estimated value of a region of idea space, you could try to:

  1. Come up with a name for the unlikely finds, even if you’ll probably update it later, as it helps confer weight to the region. In my case, I opted for normative magnets, because I thought the attractor-repeller pairs might exhibit field lines of force exerted on a model during training. I later realized an electric field is a better metaphor than magnetic (you can have single charges, you can’t have unipolar magnets), so magnets became dipoles. Normative became the more specific deontic (refers to the approach to ethics based on a set of moral rules), which then later became the more general epistemic. Despite the later iterations of the term, it helped put a finger on the region from day 1.
  2. Come up with imagery depicting the unlikely finds, as this type of identity also gives weight to the region. In my case, I opted for said field lines between P and ¬P, somewhat typical of magnet-like stuff. Here’s a picture off the whiteboard.
  3. Relate it to other notions yourself or by using thoughtware. Interweave the new fuzzy region with more established ones. Expand a mind map outwards from it by hand or via assisted recall. In my case, it happened to remind me of flashcards which aim to entrench in the learner a mental reflex from the front of the card to the back side. It also happened to relate to the various levels on the structure-function continuum, as you can have dipoles at different levels. More on that in next project’s write-up.
  4. Imagine it’s the status quo in a piece of fiction. If this idea was the theoretical foundation of a scientifically-literate society in a work of fiction, what other notions would likely be there? Fiction feels useful both in exploring specific unlikely regions, but also as a general exercise in considering unlikely regions in the first place.
  5. Put together a write-up or article about the region. See upcoming project, though you should be careful not to mistakenly make auxiliary content the object of your work if your actual focus is finding gems. I have a rule against writing articles more frequently than once a week, which might change to two weeks, so that it doesn’t actually distract me from my concrete interests above.
  6. Hate to sound like this, but create space in your calendar for freely pursuing the promising region. Blocking a time slot in your day for simply reflecting on a region of idea space feels surprisingly substantial. I pick one off the list for an hour-ish every working day, indoors by my whiteboard and laptop or outdoors if I’m fluent enough with the ideas to play with them mentally.

But how would you get to those unlikely finds in the first place, before setting camp and actively exploring the surroundings? I’d say the answer is just to read broadly, something I also struggle with, like most. You need a somewhat different personal PDF than the aggregate one, possibly also than other personal ones. I’m pretty sure the normative magnets idea came from looking up how electric and electromagnetic fields relate to each other for a course on electronics, which somehow lingered on my mind when wondering about alignment. As mentioned before in conversational multiverses, I believe concepts which are active together, wire together, leading to unlikely offspring.

When talking about reading broadly, subscribing to diverse people is only half the story. The other half is unsubscribing whenever you feel you’re quickly approaching their worldview at dangerous speeds, so that you can maintain a somewhat different personal PDF. This happened to me consciously a few times, with Scott Alexander, Bret Victor, and Yannic Kilcher. It’s not that they’re bad creators. On the contrary, they’re too good, to the point that I felt too compelled and resorted to unfollowing their work and derivatives, so I can better think for myself. But beware, trying to avoid certain worldviews is a worldview in itself, the protagonists in Unstable Orbits In The Space Of Lies are on an attractor themselves.

Also, I started following a new practice inspired by ideas related to speculative praxis. Namely, I’m implementing a personal Apert: batching all my content consumption and creation (e.g. writing this) in one day per week (currently Saturday). The rest is devoted to pursuing the stochastic winds of chance across the space of ideas and temporarily doubling down on promising regions.

There’s also something to say about framing ideation as the process of running into ideas, rather than actively thinking them, though that’s a topic for another time.