Basic Array Deduplication Using Set

For primitive-value arrays, Set provides an elegant and native solution:

const input = ['Zhang San', 'Zhang San', 'Three Zhang San'];
const uniqueSet = new Set(input);
console.log([...uniqueSet]); // ['Zhang San', 'Three Zhang San']

Object Array Deduplication via reduce()

For arrays of objects where uniqueness is deetrmined by a specific key (e.g., id), reduce() is commonly used. It enables conditional accumulation without modifying the original array的过程中.

const users = [
  { id: 0, name: 'Xiao Ming' },
  { id: 1, name: 'Xiao Zhang' },
  { id: 2, name: 'Xiao Li' },
  { id: 3, name: 'Xiao Sun' },
  { id: 1, name: 'Xiao Zhou' },
  { id: 2, name: 'Xiao Chen' }
];

const deduped = users.reduce((acc, curr) => {
  if (!acc.mapBy.hasOwnProperty(curr.id)) {
    acc.mapBy[curr.id] = true;
    acc.result.push(curr);
  }
  return acc;
}, { result: [], mapBy: {} });

console.log(deduped.result);
// Output:
// [
//   { id: 0, name: 'Xiao Ming' },
//   { id: 1, name: 'Xiao Zhang' },
//   { id: 2, name: 'Xiao Li' },
//   { id: 3, name: 'Xiao Sun' }
// ]

Using reduceRight for Priority Control

When higher-indexed duplicate entries should take precedence, reduceRight() is ideal:

const configs = [
  { name: 2, state: true, output: 'Y' },
  { name: 3, state: true, output: 'A' },
  { name: 5, state: true, output: 'S' },
  { name: 7, state: true, output: 'B' },
  { name: 3, state: false, output: 'A' }
];

const seen = {};
const latestOnly = configs.reduceRight((acc, item) => {
  if (!seen[item.name]) {
    seen[item.name] = true;
    acc.unshift(item); // Preserve original order using unshift
  }
  return acc;
}, []);

console.log(latestOnly);
// [
//   { name: 2, state: true, output: 'Y' },
//   { name: 3, state: false, output: 'A' },
//   { name: 5, state: true, output: 'S' },
//   { name: 7, state: true, output: 'B' }
// ]

Filter + Map-based Deduplication

Alternatively, filter() combined with a Map key ensuresutan efficiency and immutability:

function distinctBy(list, key) {
  const keySet = new Map();
  return list.filter(item => {
    if (!keySet.has(item[key])) {
      keySet.set(item[key], true);
      return true;
    }
    return false;
  });
}

const students = [
  { id: 1, name: 'Zane' },
  { id: 1, name: 'Zane' },
  { id: 2, name: 'Zouli' }
];

console.log(distinctBy(students, 'id'));
// [
//   { id: 1, name: 'Zane' },
//   { id: 2, name: 'Zouli' }
// ]

Key Notes on reduce()

• Supplying an initial value sets the accumulator’s starting type and state; omitting it delays the first callback invocation and uses the first array element as the accumulator.
• reduceRight() processes elements in reverse, useful for priority-based logic or preserving order with unshift().

Limitation of Set for Objects

Set performs reference comparison only, so identical object literals are not deduplicated:

const setObjs = new Set([
  { id: 1, name: 'A' },
  { id: 1, name: 'A' }
]);
console.log([...setObjs]);
// Both objects retained due to distinct references

Dropping Duplicates Across Two Arrays

To filter out objects in one array whose id exists in another:

const listA = [
  { id: 1, val: 'alpha' },
  { id: 2, val: 'beta' },
  { id: 3, val: 'gamma' }
];
const listB = [
  { id: 2, val: 'beta_dup' },
  { id: 4, val: 'delta' }
];

const idsToRemove = new Set(listB.map(item => item.id));
const filtered = listA.filter(item => !idsToRemove.has(item.id));

console.log(filtered);
// [ { id: 1, val: 'alpha' }, { id: 3, val: 'gamma' } ]