I'd be calling my pest control guy.
Get a kitty! Lol but seriously I'd call someone
I wouldn't use poison because if they crawl into a wall and die...the smell will be awful and hard to get the bodies.
Honestly, I think your best bet is to get a pest control company to come out or go to a Tractor Supply store and buy some traps. We have a rat outside that we are trying to trap. The rat is super smart and has been able to beat the trap before it springs. We will probably have to call someone out soon to come deal with the rat.
Googling now. They got to go the scratching is freaking me out!