WP All Import Duplicate Detection: Getting It Right Every Time

Duplicate detection in WP All Import determines whether a re-import creates new posts or updates existing ones. Getting this right is the difference between a clean database and one full of duplicates. The system is reliable once you understand how it works.

How Unique Identifiers Work

When you run an import, WP All Import stores a mapping between each record’s unique identifier value and the WordPress post ID it created. On re-import, WP All Import looks up each incoming record’s identifier in that mapping. If found, it updates the existing post. If not found, it creates a new post.

The unique identifier is set in the import configuration screen. It must be a value that:

• Is present in every record
• Never changes between imports for the same item
• Is unique across all records in your source file

Choosing the Right Identifier

For product feeds: use the product SKU or manufacturer part number – these are stable identifiers that do not change when the product name or price changes.

For WooCommerce products: map the identifier to both the WP All Import unique key AND the WooCommerce SKU field. This lets WooCommerce also identify the product independently.

For posts from a CMS export: use the original CMS post ID, not the title. Titles change; IDs do not.

Bad identifier choices: the post title (changes), the price (changes), the URL (changes), a generated counter (resets if you change the source).

Setting Up the Identifier in the Import

In the import configuration, go to the Unique Identifier step. Enter the XPath or column reference for your stable ID field:

[sku]           // CSV column named "sku"
{sku[1]}        // XML element named "sku" - note the curly braces
{product/@id}   // XML attribute "id" on a product element

If your source does not have a stable ID, generate one from stable field combinations:

// Combine brand + model number as a composite key
return md5(strtolower(trim([brand])) . '_' . strtolower(trim([model_number])));

Need this built properly? Describe the project and get a free estimate.

What Happens When Identifiers Change

If your source file changes the identifier for an existing record, WP All Import treats it as a new record and creates a new post. The old post remains unless you configure “Delete posts that are no longer present in your file” in the import settings.

This is the most common cause of unexpected duplicate posts: a feed provider changes their product ID system, which looks like all new records to WP All Import. Always check with your data source whether IDs are stable before using them as unique identifiers.

Deleting Posts No Longer in the Feed

When products are discontinued or posts are removed from the source, you typically want them removed from WordPress too. Enable “Delete posts that are no longer present” in the import settings. This removes posts whose identifiers were in a previous import but are absent from the current one.

Be careful with this setting when you have manual posts in the same post type. WP All Import only deletes posts it originally created (those in its identifier mapping), but verify this applies to your specific setup before enabling on a production site.

Debugging Duplicate Posts

If you are getting duplicate posts despite having unique identifiers set, check:

• Is the identifier value actually unique in the source file? Export the ID column to a spreadsheet and check for duplicates.
• Is the identifier referencing the correct field? Use WP All Import’s preview mode to see what value the identifier expression resolves to for each record.
• Are you running multiple imports of the same data? Each import has its own identifier mapping – if you imported the same file twice as two separate import configurations, each creates its own posts.
• Was the import mapping ever changed? Changing the unique identifier expression after initial import breaks the mapping – existing posts are no longer found under the new identifier value.