At a Loss for Words

Researchers at NACLO Labs are working on machine-to-machine communication, and they need your help with something! They are trying to program two robots to talk to each other about cards with shapes on them. Here are some shapes they are using:

The two robots are called Describo and Selectrix. Describo's job will be to choose a card at random and describe it. Selectrix's job will be to select the correct card that Describo described. It would be easy for Describo to describe each card uniquely. For example, it could say “filled circle”, “unfilled circle”, and “unfilled square.” But NACLO Labs wants the descriptions to be more efficient. They want the robots to follow something called Rational Speech Acts theory. This theory says that a rational speaker, when deciding what to say, will consider both the cost of their utterance as well as the likelihood that they will be understood. To make matters simple, we will say that the cost of an utterance will just be the number of words it contains: Saying nothing has a cost of zero; saying “filled”, “unfilled”, “square”, or “circle” has a cost of one; and saying “filled circle”, “unfilled circle”, or “unfilled square” has a cost of two.

Now the question is how Describo can make the shortest utterance that Selectrix can understand. If Describo wants to refer to the filled circle, it could say “filled circle,” but it could instead just say “filled” because there is only one filled shape. Saying “filled” costs less than saying “filled circle,” and Selectro can still understand it correctly. Therefore, it would make sense for Describo to say “filled” instead of “filled circle.”

G1. Here is a question to ponder: Suppose that, instead of machines, you and one other (rational) person had to communicate in this way, which is logical but not normal for humans. The other person is Describo and you are Selectrix, and the two of you are looking at the set of three cards shown above. The other person told you to choose the “circle.” Which object does the other person want you to pick? (Remember, the other person is using the Rational Speech Acts theory).

Now you want to apply Rational Speech Acts theory to a larger set of cards:

From left to right, these five cards show: a filled triangle, a filled circle, an unfilled circle, an unfilled square, and a striped square.

G2. Describe each shape in a way that has the smallest cost while remaining understandable. Your descriptions should come from the list below. You should provide each answer by writing a number from 1 to 11 next to each shape on your Answer Sheet. If multiple descriptions are tied for having the smallest cost while being understandable, you should only write one. If your descriptions were given to a (rational) listener, they should be able to understand exactly which card each description refers to—so make sure to strike the proper balance: you need to be understood, but you should also minimize the cost while doing so!

1. “filled”
2. “unfilled”
3. “striped”
4. “circle”
5. “triangle”
6. “square”
7. “filled triangle”
8. “filled circle”
9. “unfilled circle”
10. “unfilled square”
11. “striped square”

Now, to push the Rational Speech Acts theory even further, you look at an even bigger set of cards:

Reading left to right and top to bottom, these nine cards show: (a) a filled triangle, (b) an unfilled triangle, (c) a striped square, (d) an unfilled square, (e) a filled square, (f) a striped circle, (g) a filled pentagon, (h) a spotted square, and (i) a spotted hexagon.

G3. How would you describe each of these shapes with the shortest possible utterance that will still be understood? Record each description on your answer sheet by selecting a description from the list below and writing its number next to the shape it would describe. Note: A pentagon is a five-sided shape, and a hexagon is a six-sided shape. If multiple descriptions are tied for being the shortest one that will be understood, you should only write one.

1. “filled”
2. “unfilled”
3. “striped”
4. “spotted”
5. “triangle”
6. “square”
7. “circle”
8. “pentagon”
9. “hexagon”
10. “filled triangle”
11. “unfilled triangle”
12. “striped square”
13. “unfilled square”
14. “filled square”
15. “striped circle”
16. “filled pentagon”
17. “spotted square”
18. “spotted hexagon”

Now we fast-forward several years. Researchers at NACLO are looking through an abandoned lab when they find two robots, along with three envelopes containing cards. The robots had nametags saying “Descriptron” and “Identylizer”, and the envelopes were labeled SMALL, MEDIUM, and LARGE. When they looked in the envelopes, they found that they contained the same cards that NACLO Labs had been working with! The workers at the abandoned lab must have been spying on NACLO to steal some of NACLO's research. This made the NACLO researchers curious: What if these researchers made more progress on machine-to-machine communication than NACLO had? NACLO Labs assigns you to investigate.

You experiment with Descriptron and Identylizer to find out what they are capable of. You observe that Descriptron does the following:

• It starts by choosing a card at random.
• Sometimes, it then says nothing.
• Other times, it names one property of the shape on the card it chose. For example, it might say “circle”, “square”, “filled”, or “unfilled”.
• Still other times, it names two properties of the shape it chose. For example, it might say “filled circle”, “unfilled circle”, or “unfilled square”.
• Whether it names zero, one, or two properties is random.
• It never says anything wrong. For example, it never says “square” when describing a circle or “filled” when describing something unfilled.

You then notice that Identylizer could respond to what Descriptron says by picking up cards. But it doesn't always pick the card that Descriptron was describing. In order to better understand what is going on, you decide to focus exclusively on the smallest set of cards—all of the rest of the questions will be about this set of cards, which we have repeated here:

Using these three cards, you make many observations and record them in a table. Each row of the table represents what happens over multiple times when Descriptron chooses the card in the first column and says the thing in the second column. Remember that Descriptron does not always say the same thing in response to the same card. The third column shows Identylizer's error rate (the percentage of times it chooses the wrong card when Descriptron chose the card in column 1 and said the thing in column 2). To be certain that you are getting good observations, you make sure you have seen each pair of a card and a description many times.

For example, the table shows that every time Descriptron chooses the filled circle card and says “filled”, Identylizer always picks up the card with a filled circle. Its error rate is zero. Every time Descriptron chooses unfilled circle and says nothing, Identylizer only picks the “correct” card (the unfilled circle) one third of the time. Its error rate is 0.67.

G4. Some cells in the table have been left blank. Use the examples that have been provided to figure out what strategy Identylizer uses to identify a card, and then fill in the missing cells in the table based on that strategy. Each of your answers should be one of the following values: 0.00, 0.33, 0.50, 0.67, 1.00.

You show your table of data to Descriptron. After it looks at the table, a button starts flashing. The button has the word “Loss” printed on it in bright red letters. You push the button. Now, when you show Descriptron a card, it whirs, then says zero, one, or two words as before, but it also says “weight”, “error rate”, “cost”, and “loss” followed by some numbers. You notice that the values of these numbers can be different each time, so you record these outputs in the table below:

G5. Loss can be computed based on the other outputs. Describe how it is computed by filling in the blanks in the equation below. There are two correct ways to fill in the blanks; you should only pick one (it doesn't matter which one). Each blank should be filled with one of the following terms: weight, error rate, or cost.

Loss = + ( × )

You notice that Identylizer has a button on it labeled Rational Speech Acts. You push the button. Then you rerun your experiments from before (when you were computing Identylizer's error rate). But this time you get different results, as shown in the table below:

Suddenly you hear another loud whirring, and then Descriptron says “I HAVE DETECTED THAT IDENTYLIZER IS USING RATIONAL SPEECH ACTS MODE. RECALIBRATING... RECALIBRATING…” Then, after a pause, Descriptron says, “PLEASE SELECT A WEIGHT.” Seven buttons start flashing, labeled 0.1, 0.5, 0.7, 1, 5, 10, and 100. You push the button labeled 1. Descriptron then prints out an index card with the table below on it:

After printing the table, Descriptron says “SWITCHING TO BEST-DESCRIPTION MODE.” To figure out what that means, you show Descriptron the card with the unfilled square. Descriptron says, “THE BEST DESCRIPTION FOR THIS ITEM IS: <SAY NOTHING>” After experimenting some more, you figure out that the way Descriptron selects the best description is by finding the description that has the lowest loss. If there is a tie for the lowest loss, Descriptron chooses randomly between the tied options. The reason why it said “<SAY NOTHING>” for the unfilled square is that, in the table above, the option of saying nothing (written as “”) has a loss of 0.67, which is lower than the loss for the other options (2.00, 1.00, and 2.00).

G6. Currently, with the weight set to 1, Description says that “<SAY NOTHING>” is the best description for all three objects in the table above. However, by changing the weight, it is possible to get Descriptron to produce a one-word description for all 3 shapes. Circle all of the following weight values that would accomplish this. You can circle zero, one, or more than one values.

a. 0.1    b. 0.5    c. 0.7    d. 5    e. 10    f. 100