I do not think there are significant (or any) security risks associated with a user not having an address book record.
Your problem will be a loss in functionality. Off the top of my head users missing an address book record will be unable to use the Work Center and will not be able to participate in Approval Routing which relies on Address Book Numbers to define the approval hiearchy. I would also think that would affect some of the workflow processes. Even if you believe you are not using workflow, you are probably using it in some fashion as many of the core OneWorld application reference pre-defined workflow processes.
Regards,