Why shouldn't I just use the key?

Because it's unverified. You have no way of knowing that some third party isn't pretending to be Tom to fool you. To verify the key, you should collect it and extract its fingerprint. Then you should check the fingerprint, keyid and length.

You can do this by calling me directly at work (+44/0 1223 704038) and asking me to verify the fingerprint, id and length.

If you don't understand why an unverified signature is bad, you probably shouldn't be using PGP.