They guy was using slightly different email address each time. He was never confirming the account but his names would show up on the newest member part of the statistics box. 

I looked for his IP in the debug section but his account were empty and had no info. 

I managed to find his IP address on a site called Stop Forum Spam and then blocked it in my server. I think this should fix the problem, and he shouldn't be able to get onto my site at all now. 

I will let you know if he returns. 

Works well for me: https://wordpress.org/plugins/stop-spammer-registrations-plugin/

Works most of the part but extrememly heavy.

The problem is, when they sign up their username shows as newest members even if the account isn't activated. If a user didnt show on the newest members section of the footer until they confirm their account that would be cool :) 

You can ban an IP via Cpanel of your account or easily via .htaccess.

If so, check BanHammer plugin.

https://wordpress.org/plugins/ban-hammer/

How can I stop this from happening? Is it possible to ban the email address? 

And if a member is banned from the forum, surely that email address should not be able to make an account again? 

Any help is massively appreciated :)