Removing duplicates from a data set the php way
Aug 02, 2010
by
0
There are times when you receive some data that has duplicate records that are throwing a wrench in your programming. You may only want the distinct list items from the data source. There are a few ways you can accomplish this in php. You will need some way to uniquely identify an instance from your data set. When your data set is a simple list, that unique identity can be as simple as the list row itself.
You can use the associative key of an array to store items in the list, this will prevent duplicates as only one unique key will be set in the array. You could also use a simple array and perform a check to verify that the item does not exist before inserting in array. The example below shows the associative array method for preventing duplicates.
In this example, the "text" variable is the data source, it will be broken apart by the line break character.
$out = array();
$lines = explode("\n",$text);
foreach($lines as $l) {
$item = trim($l);
$out[$item] = $item;
}
It is as simple as that, the same methodology can be applied to any type of data. Just determine a unique identifier for the record, then check if that exists or use an associative array to prevent duplicates.
Leave a comment
To leave a comment, please log in / sign up
RSS Feed
Facebook
Twitter
LinkedIn
YouTube