If you meet with this person I can only suggest speaking to them about the matter in person. If they are a pen friend, simply convey your feelings to them through the same means that you would hold a conversation with them.
Simply let the person know that you don't wish to speak to them, and give your reasons so that they aren't left to wonder - I know a thing like that would drive me crazy.
If they persevere, remind them that you don't wish to speak to them, at which point you separate yourself from them. You shouldn't hold yourself accountable for how the recipient responds as everyone reacts differently to these situations.
_________________
I used to plan and plot, and try to live correct; lately I do a lot of things that don't make sense. Now I must do what I must do.