Connecting your local contacts to Tact is necessary to import the Exchange, Google and local contacts from your device to your Tact account. All this customer data will be de-duplicated within Tact!
Although you may have personal contacts from your device that show up in Tact, it's not necessary to remove them. These contacts are never automatically synced to Salesforce. They would only sync to Salesforce if you logged an activity or added a task or note related to the contact.