Money is not a problem. The general risk is transferring to a stranger for fraud. Going to the bank counter will require you to provide the information of the person you are transferring money to, and then call them to confirm why this money is being transferred to you. After you resolve this, your card may be compromised.