Completed

Duplicate Detection

Would be great if Insightly had a filter that showed all the names that match in the system along with a bulk merge option. Any plans for this?

70

Comments

36 comments
  • Official comment

    Hello everyone, we are happy to announce that SmartMerge is now available in Insightly for Contacts and Leads as well as Organizations.

    With SmartMerge you can quickly find and clean up duplicate records by merging them into one master record.

    The attributes that are merged for contacts are: links, tags, tasks, notes, emails, file attachments, contact infos, dates and addresses. The attributes that are merged for leads are: tags, tasks, notes, emails, file attachments and addresses.

    Learn more about dealing with duplicates in our help guide and let us know what you think about SmartMerge for Contacts & Leads in the comments area below. 

    Comment actions Permalink
  • I am in the process of cleaning up duplicate data in the system, and I was wondering if this is something that can be done automatically?  Can we merge the duplicate contacts and other entries automatically, or do I have to go one by one?

    3
    Comment actions Permalink
  • I believe it's one by one right now. If there is a way to bulk merge, we haven't found it. :(

     

    3
    Comment actions Permalink
  • Yes, I'm also having lot of time spent to check for doubled (maybe slightly mistyped) entries of organizations and contacts with my team ...

    It's really painful please add a feature that can help to reduce this scrap time .... some pop-up with indication of possible duplicate immediatly after the input with an option to merge would be great.

    4
    Comment actions Permalink
  • Agreed. The last system I worked on had built in duplicate warnings.

    So anytime a person or organization was entered that matched records already on file (ex matching email address, matching name, or matching phone number) a warning would be generated that the information matched another record already in the system (with a link to the record/records that would open in a new tab so that you can see if it was actually a duplicate). You'd have to click 'override' to then enter the record if it triggered a duplicate warning. 

    5
    Comment actions Permalink
  • Also, now that we're all using the lead management the import leads should have a check for duplicated contacts/organizations

    2
    Comment actions Permalink
    • 1
    1
    Comment actions Permalink
  • I would love the opportunity to merge duplicate opportunities with the consolidation of all notes, emails and tasks.

    1
    Comment actions Permalink
  • I have found that when a Contact or Org's name is spelled EXACTLY the same way it is spelled in Insightly (including spaces and punctuation) it merges, otherwise there is a duplicate. I have added information to Orgs (i.e. web address, DID phone number) by importing spreadsheets, and have had to copy and paste the Org name from Insightly into the corresponding cell on the spreadsheet for it to work. Other programs I have used (that cost more than $7.00/month, BTW) have been more intuitive than that.

    0
    Comment actions Permalink
  • Hi everyone,

    My name is Chris and I am on the engineering team.  I am interested in hearing more of your thoughts, ideas, and concerns regarding this issue.  That way we can put together a plan to address this issue in the best manner possible.

    Thank you for being active in our community!

    4
    Comment actions Permalink
  • Thanks for your interest Christopher.

     

    Ideally, duplicate detection in insightly could be implemented in two complementary flavours:

    1) "On the spot" when entering new contacts manually: when **some** of the information entered for a new contacts (like email address or name / family name) is entered, automatically detect similar entries and suggest merging the information (Android smartphones do this when entering new contact information quite effectively).

    2) "On demand": have a function in insightly that detects **similar** entries and suggest merging their information with an easy to use interface.

    Example of the result for an "on demand" query (I'm not a republican, just for the sake of the argument):

    1) George Bush (gwbush@bush.com)

    2) George Bush (ghbush@bush.com)

    3) George Bush (gbush@gmail.com)

    4) George W. Bush (gwbush@bush.com)

    5) George H. Bush (ghbush@bush.com)

    These 5 entries are similar and presenting additional information such as email next to it (could be more if it fits) allows one to easily sport which "George Bush" we are talking about. In my ideal user interface, I could tick (1), (2) and (4) a click "merge" and then tick (3) and (5) and click "merge" and end up with only two contacts with all the information and links contained in the originating contacts put together.

     

     

    3
    Comment actions Permalink
  • Christopher I totally agree with jmd plus a couple of feature that can help when you already have a populated database:

    1) We started using Leads after having populated the contacts and organizations (just because in our plan Leads have been recently added), so both flavours should check within the 3 groups: Leads, Organizations, Contacts (for example to prevent adding a Lead when the same Organization already exists).

    2) In both cases "On demand" or "On the spot", when resolving Leads/Contacts duplication I should be able to choose the right Organization among the **similar** entries, example:

    • Murder Inc.

    • Murder Inc

    • Murder

    • Murder Incorporated

    • M*rder - Daytona office

    As general comment the CRM should be useful also for backoffice staff who actually may don't know too much about the business and if they have to register an incoming call or email request they should be immediately able to check if it's a new lead or an existing organization.

    I hope this comment may help you Chris and wish you good success with the task

    4
    Comment actions Permalink
  • Opps, choose and by thicking more than one, merge the organization also ....

     

    0
    Comment actions Permalink
  • I initially imported a mailing list into Insightly and recently imported additional information about existing orgs/contacts gained by additional research such as websites, email addresses, etc. Duplicate addresses were created when there was any variation at all, such as when phone numbers were formatted differently, and when the same addresses varied by using "Suite" vs. Ste", or Zip Code versus Zip+4. Also, Please include a provision for Middle Name/Initial and Name Suffix (I call on a lot of Doctors) , since these elements should be present when using Insightly for doing direct mail, etc., and Custom Fields are required now. When importing additional information about an existing contact, the confirmation email only reports that the new Orgs were imported, but

    0
    Comment actions Permalink
  • Plus one vote for this feature! 

    1
    Comment actions Permalink
  • Add me too! 

    1
    Comment actions Permalink
  • As long as we're all talking, hard fields in Contacts for Middle Name/Initial and Suffixes (i.e. Jr., MD) would be appreciated and would be a factor in Duplicate detection. I created Custom Fields for this (I deal with a lot of Doctors), but it's a poor substitute.

    2
    Comment actions Permalink
  • It is frustrating when Insightly brings the email company in without checking if a similar organisation exists.That way we have loads of duplicates.

    Any suggestions to limit this would be great.

    4
    Comment actions Permalink
  • Here is a solution, and it will cost you one way or another!

    There is an integrated 3rd party software that identifies duplicated contacts by matching emails and first and last names.

    The software also merge contacts that are exactly the same.

    It's called Dedupedly.

    Dedupedly merges all duplicate contact info including avatar, social accounts, emails, addresses, notes, tags and custom fields. In addition Tasks, Events, Notes, Emails, Projects and Leads are re-assigned to merged contacts.

    Hope this helps!

    2
    Comment actions Permalink
  • +1 Vote for a dedupe feature in Insightly

    2
    Comment actions Permalink
  • We are having a really tough time getting out duplicates for a marketing email we will be sending out. We are trying to merge LinkedIn contacts with those that already exist in the Insightly database. We have compared by first and last name and email address, but often LinkedIn email addresses are personal emails, while the emails we have in our Insightly database are professional emails. It would be amazing (and as stated above) if Insightly could recognize emails that were similar. Or if there could be a report run for contacts who match by first and last name so that they can be inspected to see if they are indeed a duplicate or not. 

    0
    Comment actions Permalink
  • Sorry to hear about the troubles you are having @Jennifer. Thanks for chiming in, it helps to see how this tool can be useful in different day to day scenarios for our users.

    1
    Comment actions Permalink
  • Right now we're setting up Insightly for us and therefor try to just avoid duplicates, which is not easy. Any helpful suggestions?

    We plan to import our cases as Leads. But they're not created as Organizations or Contacts until we convert them into Opportunities, if I got it right. So how do I even search for duplicates by Organization, since Leads are listed by Contacts?

    One way to easier find these would be to insert the data as Organizations as well as Lead. But what happens to the Org.-entry when I convert the Lead to Opportunity?

    0
    Comment actions Permalink
  • Hi @Johannes, You may find unique ways to use Insightly that fit your personal workflow.

    There is duplicate detection when importing Contacts and Organizations, if this works for you. 

    When you convert your Leads an organization will only be created if an organization name is included in the Lead and doesn’t already exist.

    1
    Comment actions Permalink
  • Hi,

    A quick update on this feature request. We chose to tackle it in a few different ways.

    We have now rolled out duplicate detection when entering a new contact or organization (http://support.insight.ly/hc/en-us/articles/212680777-Duplicate-detection) so users entering in new contacts or organization forms manually can see in real time if those records or very similar records already exist in Insightly. 

    We're also in the final stages of testing our new smart merge feature. This is a very advanced fuzzy matching algorithm that finds exact duplicates or very similar spelling or phonetically sounding records to your existing records, and allows you to merge those records together into one record in just a couple of clicks, giving you the options of which fields to keep and which to override, which record to designate as the master record and which to delete, and the ability to merge several duplicate records at once  into one master record.

    Finally we're also producing a system report that you can run that deeply analyses all the contact and organizations in your Insightly, and finds and suggests records it identifies are potential duplicates. From that report you can then merge those records together in just a couple of clicks. 

    I'll update you again when those last two features ship in the very near future.

    Thanks,
    Anthony.

     

    2
    Comment actions Permalink
  • Thanks, I noticed this feature recently and it's fantastic!  Great work.

    0
    Comment actions Permalink
  • Thanks. I appreciate the new tool. Hopefully the smart merge and duplicate report will help in cleaning up database!

    0
    Comment actions Permalink
  • Great ask; there is a great Open Source product known as Duke (https://github.com/larsga/Duke) that uses a relatively sophisticated BBN as a means to determine record duplication in a variety of ways, to include name comparisons, address comparisons (including geo-proximity!) and others! I highly recommend you take a look at this approach for this critical function!

    0
    Comment actions Permalink
  • We've now shipped the second piece of our duplicate detection and merge system with the new smart merge feature and the duplicate email report. You can read about the smart merge in our help article at:

    http://support.insight.ly/hc/en-us/articles/214937878-Running-a-SmartMerge-report-for-organizations

     

    0
    Comment actions Permalink
  • You need to allow bulk merges. It takes forever to do manually. You should show a screen with differences and then allow selection of records to be merged at once. 

    0
    Comment actions Permalink

Please sign in to leave a comment.