Extending Conceptual Thinking Through Semantic Embeddings

Turning numerical quirks into powerful tools for thought.

  • Codename

    Semantica

  • Stage

    Proof of Concept

Open Project

Mental models are simplified descriptions of the world around us. For instance, one of them might describe networks. A forest is a network of trees. A society is a network of people. A brain is a network of neurons. Mental models help us make sense of the world by allowing us to apply previous knowledge to new situations. They are widely seen as powerful tools for thought, especially when they come in large numbers. If one’s repository of mental models is vast, then they’ll be able to approach new situations from many different perspectives. This is the motivation behind many recent efforts of compiling extensive lists of them.

Our systematic cross-realm translations are the roots of fruitful metaphors; they enable us to understand things we’ve never seen before. When something seems entirely new in one of our description-worlds, it may turn out that when translated to some other world it resembles something we already know.

— Marvin Minsky

However, mental models are only one side of what can be more broadly described as conceptual thinking. In this view, mental models are just sets of systematic relations between concepts. The previous network model merely captures the relation between a forest and a tree, between a society and a person, and between a brain and a neuron. Having said that, there is so much more to concepts than mental models. You can connect them to similar ones. You can mix them together into new ones. You can transform them in meaningful ways. You can explore the nuances between them.

What if we could build tools which enabled us to work with concepts in a similar way Photoshop enables us to work with images? What if we could build tools which extend our conceptual thinking beyond what is humanly possible? Instead of blending colors, we would combine concepts. Instead of creating gradients, we would explore continuums of meaning. Instead of defining intricate visual patterns, we would define systematic patterns of meaning.

In this it resembles a program such as Photoshop or a spreadsheet or 3D graphics programs. Each provides a novel set of interface primitives, primitives which can be internalized by the user as fundamental new elements in their thinking.

— Michael Nielsen

However, tools like Photoshop don’t directly work with colors, gradients, or patterns. At the lowest level, editing photos boils down to manipulating matrices of numbers. In order to build powerful tools for conceptual thinking, we might need an analogous way to fix concepts into firm numerical foundations which we could then easily manipulate.

Fortunately, there already are ways of doing that. The field of natural language processing has long used semantic embeddings as the numerical substrate of discrete concepts. Among others, they’re used in search engines to understand queries, in chatbots to understand conversations, and in translation systems to understand foreign languages. Think of semantic embeddings as numeric coordinates. They don’t describe locations in a physical space, like geographic coordinates, but locations in a space of meanings, a semantic space.

An intuitive understanding of how semantic embeddings are obtained is beyond the scope of this article, but what is relevant for our current purposes can be captured in a few neat properties exhibited by the semantic space:

1. Conceptual differences correspond to geometric distances.

2. Conceptual parallelism corresponds to geometric parallelism.

But algebraic geometry is no reason for despair, because as graphic designers don’t need to be knowledgeable about convolutions and tensors when using Photoshop, the tools for thought which we set out to build will be usable regardless of the user’s proficiency in maths. We’ll use semantic embeddings only as a low-level foundation for higher-level tools which enable anyone to work with concepts in exciting ways.

That’s where we’ll go next. In the following sections, we’ll define and use new tools for thought built on top of semantic embeddings, and in doing so incrementally grow Semantica, a veritable computational toolkit for conceptual thinking.

01

Field

Functional Description

Finds concepts which are closely related to a given concept.

Spatial Intuition

Finds concepts which are close to a given concept.

Numerical Implementation

Finds concepts whose embeddings are the most similar to the embedding of a given concept.

>>> field(‘car’)

[‘vehicle’, ‘cars’, ‘suv’, ‘minivan’, ‘truck’, ‘ford_focus’, ‘honda_civic’, ‘jeep’]

>>> field(‘galaxy’)

[‘galaxies’, ‘milky_way’, ‘planets’, ‘supernova’, ‘galactic’, ‘universe’, ‘comet’, ‘planet’, ‘cosmos’]

>>> field(‘bed’)

[‘beds’, ‘couch’, ‘sofa’, ‘sleep’, ‘duvet’, ‘sleeping’, ‘bunk’, ‘pillow’, ‘mattress’]

A semantic field is a set of words related in meaning. This tool can be used to expand concepts into their semantic fields.

02

Mix

Functional Description

Blends given concepts into new ones.

Spatial Intuition

Finds concepts which are close to the center of the given concepts.

Numerical Implementation

Finds concepts whose embeddings are the most similar to the average embedding of the given concepts.

>>> mix(‘people’, ‘chaos’)

[‘anarchy’, ‘mayhem’, ‘chaotic’, ‘civil_strife’, ‘bedlam’, ‘strife’, ‘bloodshed’, ‘upheaval’]

>>> mix(‘computer’, ‘virus’)

[‘viruses’, ‘computers’, ‘antivirus_software’, ‘malware’, ‘spyware’, ‘worm’, ‘antivirus’]

>>> mix(‘brain’, ‘science’)

[‘neuroscience’, ‘brains’, ‘biology’, ‘physiology’, ‘cognition’, ‘mathematics’, ‘neural’, ‘cognitive’]

Conceptual blending has been described as the process of projecting multiple concepts onto a blended mental space. If this explanation seems circular, that’s because it is. Still, this tool can be used to perform this ill-defined but intuitive task.

03

Span

Functional Description

Finds a sequence of concepts which spans the continuum between two given concepts.

Spatial Intuition

Finds concepts located along the line between two given concepts.

Numerical Implementation

Finds concepts whose embeddings are the most similar to the interpolated embeddings of two given concepts.

>>> span(‘pond’, ‘ocean’)

[‘pond’, ‘ponds’, ‘retention_pond’, ‘drainage_ditch’, ‘creek’, ‘creek_bed’, ‘lake’, ‘river’, ‘lagoon’, ‘marsh’, ‘sea’, ‘ocean’]

>>> span(‘city’, ‘house’)

[‘city’, ‘mayor’, ‘municipality’, ‘municipal’, ‘district’, ‘downtown’, ‘town’, ‘neighborhoods’, ‘neighborhood’, ‘houses’, ‘house’]

>>> span(‘kindergarten’, ‘university’)

[‘kindergarten’, ‘kindergartners’, ‘preschool’, ‘sixth_graders’, ‘eighth_grade’, ‘elementary’, ‘school’, ‘students’, ‘university’]

The selected samples are massively cherry-picked. However, in the envisioned use cases of this toolkit, there’s always a human-in-the-loop who is able to sift through some moderate amounts of noise.

04

Shift

Functional Description

Captures the relation between two given concepts.

Spatial Intuition

Determines the directed difference in location between two given concepts.

Numerical Implementation

Computes the arithmetic difference between the embeddings of two given concepts.

>>> mix(‘cell’, shift(‘biology’, ‘physics’))

[‘atoms’, ‘electron’, ‘electrons’, ‘photons’, ‘neutrons’, ‘particle’, ‘photon’, ‘physics’]

>>> mix(‘saxophone’, shift(‘jazz’, ‘rock’))

[‘rock’, ‘guitar’, ‘bass_guitar’, ‘guitars’, ‘electric_guitar’, ‘rocks’, ‘guitar_riffs’, ‘trombone’, ‘guitarist’]

>>> mix(‘burrito’, shift(‘Spain’, ‘Italy’))

[‘pizza’, ‘burger’, ‘sandwich’, ‘pasta’, ‘pizzas’, ‘cheeseburger’, ‘pizzeria’, ‘hamburger’, ‘sushi’]

“Metaphor” comes from the Latin “metaphora”, meaning “to carry over”. This tool can be used to “carry over” concepts from one domain to another.

05

Match

Functional Description

Finds sets of concepts whose elements match the relations found in a given set of concepts.

Spatial Intuition

Finds constellations of concepts which match the shape of a given constellation of concepts.

Numerical Implementation

Finds sets of concepts whose internal differences in embeddings are the most similar to the ones found in a given set of concepts.

>>> match(‘people’, ‘society’)

[‘members’, ‘membership’]

[‘players’, ‘team’]

[‘students’, ‘classroom’]

[‘women’, ‘womanhood’]

[‘customers’, ‘clientele’]

[‘workers’, ‘workforce’]

[‘fans’, ‘fandom’]

>>> match(‘physics’, ‘Einstein’, target=’science’)

[‘biology’, ‘charles_darwin’]

[‘psychology’, ‘freud’]

[‘linguistics’, ‘chomsky’]

[‘philosophy’, ‘nietzsche’]

[‘astrophysics’, ‘stephen_hawking’]

>>> match(‘king’, ‘queen’, target=’acting’)

[‘actor’, ‘actress’]

[‘al_pacino’, ‘meryl_streep’]

[‘cocky’, ‘bitchy’]

[‘best_actor’, ‘best_actress’]

[‘showman’, ‘diva’]

Inspiration for this tool comes from the science fiction trilogy “The Three Body Problem,” by Cixin Liu. There, the main character needs to broadcast the location of a celestial body to a set of unknown alien civilizations. However, given the lack of absolute reference frames available, he broadcasts the position of the celestial body relative to several neighboring bodies. Here, because the dimensions of the semantic space aren’t inherently meaningful, a mental model is expressed as a set of distances from the first concept to each subsequent concept, forming a “constellation” of concepts. The Golden Records use a similar scheme to pinpoint the Earth.

Full Toolkit

Semantica Proof of Concept

Field

Finds concepts which are closely related to a given concept.

Mix

Blends given concepts into new ones.

Span

Finds a sequence of concepts which spans the continuum between two given concepts.

Shift

Captures the relation between two given concepts.

Match

Finds sets of concepts whose elements match the relations found in a given set of concepts.

Case Studies

Graphen

Physicist & Biologist

CASE STUDY 01

My research group and I have been exploring potential applications of graphene for several years now. It’s a really fascinating material. You know, graphene is like…

>>> mix(‘graphene’, shift(‘physics’, ‘biology’))
[… ’tissue’ …]

…tissue. Graphene is like a tissue of carbon atoms, in a similar way in which biological tissue is composed of a latticework of interconnected cells. It turns out to be quite resistant, yet flexible.

antenna-jqh0GEvuNBY-unsplash

Artist & Scientist

CASE STUDY 02

We see ourselves as living in two radically different worlds, but there’s a seamless transition between them. Consider interdisciplinary fields such as…

>>> span(‘art’, ‘science’)

[… ‘humanities’, ‘museology’ …]

…humanities or museology. We can meet each other halfway through.

nathan-dumlao-xPHmmVKS8lM-unsplash

Sociologist & Students

CASE STUDY 03

Think of a society as a…

>>> match(‘people’, ‘society’, target=’student’)


[‘students’, ‘clasroom’]

…classroom, composed of many independent students who all have their own individual beliefs, desires, and intentions.

Further Steps

Friendlier interfaces

From Photoshop-like stand-alones to Wolfram-like web apps, there are exciting ways of wrapping interfaces around these conceptual tools.

Better tools

This early selection of tools merely scratches surface of how semantic embeddings can be used in building tools for thought. A largely unexplored space of possibilities is waiting for curious thinkers.

Deeper integration

This toolkit only operates with knowledge on a conceptual level. In the future, it might be able to interface with definitions (e.g. from WordNet), multimedia content (e.g. from ImageNet), external resources (e.g. via Zotero), or more established tools for thought (e.g. Zettelkasten).

Higher performance

The current software implementation has been developed for experimental purposes, rather than efficiency. Much needed code optimizations will significantly improve the speed of the algorithms involved.

Better embeddings

Not all semantic embeddings are created equal. The ones used in this prototype have been obtained through a relatively rudimentary approach. Newer techniques capture meaning more effectively and with less bias.

Contributions

The idea that semantic embeddings can be directly used to build tools for thought, rather than only as raw ingredients in downstream machine learning tasks.

The idea that mental models can be formalized as constellations of semantic embeddings in semantic space.

The formalization and implementation of Span and Match. The other conceptual operations (i.e. Field, Mix, Shift) were already formalized and implemented in earlier work, one way or another. For example, Mix is based on additive composition of semantic embeddings, a process described in 2013. However, these operations were largely used to measure the quality of semantic embeddings for downstream tasks, rather than as first-hand tools.

The strengthened link between Photoshop and more radical tools for thought, through Photoshop-like names, icons, and descriptions. Photoshop has been extensively used as a prime example of tools for thought before, but the current work explores new ways of reinforcing this connection.

The name Semantica for a tool for conceptual thinking has been inspired by the name Mathematica, used by Stephen Wolfram to describe a tool for computational thinking.

Try it out yourself

Code & Demo