---
title: "Simon Willison on X: \"Anyone got a lead on a good embedding mo..."
description: "!https://twitter.com/simonw/status/1700528222382027039"
date: 2023-09-10
published: true
tags:
  - ai
  - llm
  - thought
template: link
---


<div class="embed-card embed-card-external">
  <a href="https://twitter.com/simonw/status/1700528222382027039" class="embed-card-link" target="_blank" rel="noopener noreferrer">
    <div class="embed-card-image">
      <img src="https://pbs.twimg.com/profile_images/378800000261649705/be9cc55e64014e6d7663c50d7cb9fc75_200x200.jpeg" alt="Simon Willison (@simonw) on X — Anyone got a lead on a good embedding model that can embed both images and text into the same space, so you can search for &#34;dog&#34; and get back images most likely to contain a dog?

It looks like VisualBERT is one, what are others?" loading="lazy">
    </div>
    <div class="embed-card-content">
      <div class="embed-card-title">Simon Willison (@simonw) on X</div>
      <div class="embed-card-description">Anyone got a lead on a good embedding model that can embed both images and text into the same space, so you can search for &#34;dog&#34; and get back images most likely to contain a dog?

It looks like Vis...</div>
      <div class="embed-card-meta">X (formerly Twitter) &middot; twitter.com</div>
    </div>
  </a>
</div>


Kinda mindblown that this is even possible.  This is so far outside of my current thinking that i didn't even think of an elegant way to implement semantic search accross images and text at the same time.  I know it happens at Google, but I envision that as still text search accross tags and meta data about the image.

Based on the number of responses CLIP is the thing that does this.