Week 3: Section Activity

Section activity: When you arrive at section, you'll start with your laptops closed. Your TF will start by leading a discussion about Ted Chiang's article and your general observations about GPT-4o's paragraph completions.

Then your TF will organize a contest to select the paragraph stem that seemed to give the widest range of different completions. Once a "winning" prompt has been chosen, everyone will test that prompt 10 times in the AI Sandbox (with each student using a different LLM). Your TF will lead a discussion, collecting observations about the different LLMs.

Finally, you will explore clustering and vector embeddings in LLMs. Your TF will help you load our vector embedding code into a JupterHub notebook, following the instructions in Word_Embeddings.ipynb Download Word_Embeddings.ipynb.

Working in pairs, you will:

• make a ranked list of similar words using the similar function—can you find some surprising examples?
• find more examples of "clustering" of similar/dissimilar words using the plot2d function
• explore "conceptual arithmetic" using the analogy function

Note that the numbers associated with each word in the output are the cosine similarities—recall that a cosine of 1 means the vectors point in exactly the same direction, and the cosine gets smaller as the vectors become less similar.

Your TF will explain how the analogy function works—it depends on the idea that you can add or subtract vectors. So if you subtract (woman–man), you get a numerical vector representation of the difference between woman and man. If you then add this to a male-gendered word (e.g. king), then you get the following "arithmetic":

king + woman – man = ?

which represents the concept, "what if you start with the vector for king, and add the vector that represents the change from man to woman". That gives a new vector. Now find the word that is closest to that vector. Voilà! The answer is queen. This isn't a true equality—the vectors don't exactly add/subtract to give queen, but it is approximately true:

king + woman – man ≈ queen

If we rearrange this equation it looks kinda like an analogy:

man – woman ≈ king – queen

which expresses a sense in which "the difference between man and woman is similar to the difference between king and queen." In the language of analogies, we might say:

man is to woman as king is to queen

Try to find other examples of word analogies. You can also plot the four words from an analogy pair and see: do the vectors seem to be doing approximately what you would expect? For instance, the difference (shift) from man to woman should be similar to the difference from king to queen. (At least the directions should be similar.)