|Well, he's certainly chipper (credit)|
Since writing instruments were originally fine brushes, this lineage makes a lot of sense. Applying pencil to the modern "graphite writing implement" dates to the late 16th century.
If you noticed the similarity between Latin penicillus and English penicillin, you're in good company. Penicillin was taken from Penicillus notatum, which is the name of the mold from which the medicine derives. It got the name because its cells apparently resemble pencils.