Those scans/bruteforce attempts are mostly harmless anyways, so there is very little benefit in changing the port. Has there been yet a single drive-by 0day attack against opensshd, because that's essentially the only thing the port change would help against.
The benefit in changing the port is that if you do reporting/logging of these things, the potentially-harmful scans aren't hidden under a morass of benign dictionary attacks.
This is unrelated, but I keep getting emails from my home server every two weeks or so that someone made three incorrect sudo attempt to run "sudo sh /tmp/somefile". There's nobody logged in via SSH, I have key-only author anyway, and the user who's trying to do this is apparently me. It doesn't originate in a console, and the only thing running as me is the Dropbox daemon. Anyone have any idea what's going on, and if I should be worried?
Dictionary attacks are by definition not benign. If a compromised host is sending any packets your way, it's best to drop them on the floor where they belong. Then, if for any reason, the person controlling that other host happens to e.g. run portscans to find other services or point nessus / metasploit your way, they are going to have a much harder time of doing so.
