Engineering better search with Grant Ingersoll
In the follow-up episode of Vector Podcast Live series, I’ve interviewed Grant Ingersoll — Fractional CTO, Leading Search Consultant, co-founder of Lucidworks, former CTO of Wikimedia. Grant took the engineering stance of evolving a search engine.
We discussed some fascinating topics, like how to nurture intuition in building search engine algorithms, how to measure the success of your search engine, and role of vector search in the modern search engine development.
As usual, I’ve also asked my favorite magical question of WHY — drilling into Grant’s motivation to work on search and to teach it!
Some more topics we covered:
03:03 Grant’s background
04:58 Principle of 3 C’s in the life of a CTO: Code, Conferences and Customers
07:16 Principle of 3 C’s in the Search Engine development: Content, Collaboration and Context
11:51 Balance between manual tuning in pursuit to learn and Machine Learning
15:42 How to nurture intuition in building search engine algorithms
18:51 How to change the approach of organizations to true experimentation
23:17 Where should one start in approaching the data (like click logs) for developing a search engine
29:36 How to measure the success of your search engine
33:50 The role of manual query rating to improve search result relevancy
36:56 What are the available datasets, tools and algorithms, that allow us to build a search engine?
41:56 Vector search and its role in broad search engine development and how the profession is shaping up