In working on a post for tomorrow on whether Large Language Models like GPT-4 and Bard-2 have beliefs, I asked GPT-4 what I thought would be a not-too-hard question about chemistry: "What element is two to the right of manganese on the periodic table?" It crashed, burned, and exploded on the spot, giving two different wrong answers foot on tail, without noticing the contradiction:
The correct answer is cobalt, element 27. Here's the text of the exchange, if you can't easily read the image:
You: What element is two to the right of manganese on the periodic table?
ChatGPT: The element that is two places to the right of Manganese (Mn) on the periodic table is Iron (Fe). In the periodic table, elements are arranged in rows and columns based on their atomic number and electronic configuration. Manganese has the atomic number 25, and Iron has the atomic number 26. The element to the immediate right of Manganese is Technetium (Tc) with atomic number 43, and then comes Ruthenium (Ru) with atomic number 44.
Let me emphasize that this answer is not turd-picked (the opposite of cherry-picked). I didn't seek to stump GPT-4, and I didn't ask multiple questions looking for a bad answer. This was the first question I posed.
I posed the same question to Bard-2, and it did a little better, but still didn't quite get it right:
Me: What is two elements to the right of Manganese on the periodic table?
Bard-2: Manganese is located in period 4 and group 7 of the periodic table. Moving two elements to the right of Manganese would be to group 9. In group 9, the elements are Iron (Fe) and Cobalt (Co).
So I tried two more on Bard-2 (again not picked), first what should be an easy one, then a somewhat harder one.
Me: What other elements are in the same column of the periodic table as Chromium?
Bard-2: Chromium is located in group 6 of the periodic table. Other elements in the same column as chromium are molybdenum (Mo) and tungsten (W). [here it displays an image of the periodic cell for molybdenum] Group 6 elements are also known as transition metals. They are characterized by their incomplete d electron shells, which give them their unique properties, such as high melting points, hardness, and the ability to form alloys.
Here Bard-2 is 2/3 right, leaving out Seaborgium. (But maybe that's excusable, if transuranic elements shouldn't really count?)
Me: In the periodic table, what element is one down and one to the left of Silicon?
Bard-2: Silicon (Si) is located in period 3, group 14 of the periodic table. One down from Silicon is Germanium (Ge), and one to the left is Aluminum (Al). Therefore, the element one down and one to the left of Silicon is Aluminum (Al).
While it is true that germanium is one down and aluminum is one to the left, Bard-2 fails to compound the down-and-left movement to the correct answer of gallium.
Returning to GPT-4, I posed exactly the same two questions. Despite GPT-4's horrible performance with my first question, it did get those two questions correct.
Overall, then, two out of six. Don't use LLMs yet for your elementary chemistry exam!
ETA 3:47 pm:
GPT-4's answer to the silicon question is actually a pretty good demonstration of how an LLM can work its way to the correct answer without visual knowledge, so I'll paste it here:
To find the element that is one down and one to the left of Silicon (Si) on the periodic table, we need to look at the element directly below Silicon in the same group (column), and then move one column to the left.
Silicon is in Group 14 of the periodic table. The element directly below Silicon in Group 14 is Germanium (Ge). Moving one group (column) to the left from Germanium, we find Gallium (Ga) in Group 13. Therefore, the element that is one down and one to the left of Silicon is Gallium.
These results aren't surprising if you consider what sort of sentences you would expect to predominate in the training data. Vertical relationships are important in chemistry, but horizontal relationships aren't especially meaningful, so you would expect it to do a better job on vertical rather than horizontal movements. (I think omitting seaborgium is OK, since seaborgium compounds wouldn't be prominent in discussions of group 6 chemistry.)
The diagonal relationship you asked about, "down and to the left," is also not a chemically meaningful relationship — however, "down and to the right" (or "up and to the left") can be a chemically meaningful one:
https://en.wikipedia.org/wiki/Diagonal_relationship
I predict the LLMs should be expected to do a better job on these sorts of diagonal relationships than on on the ones you were asking about.