Properly unregister the driver if a device is removed

Fix errors occurring because the tlkm driver does not react properly on device remove requests.