Mongoose.aggregate(pipeline) koppelt meerdere collecties met $unwind, $lookup, $group

Dit was een lastige voor iemand die nieuw was bij MongoDb 's aggregate . Ik zal mijn antwoord opsplitsen in stappen om te demonstreren aan anderen die proberen een array samen te voegen met verwijzingen naar meerdere collecties.

Stap 1 - $match om te filteren op collectie

De $match accepteert dezelfde zoekopdrachten als db.collection.find({}) en retourneert een reeks overeenkomende resultaten in het onderstaande geval, ik selecteer hier 4 specifieke records


{ '$match':
     { _id:
        { '$in':
           [
              ObjectId('5f7bdb3eea134b5a5c976285'),
              ObjectId('5f7bdb3eea134b5a5c976283'),
              ObjectId('5f7bdb3eea134b5a5c976284'),
              ObjectId('5f7bdb3eea134b5a5c976289')
           ]
        }
     }
}

$match Resultaat

[ 
  { _id: ObjectId('5f7be0b37e2bdf5b19e4724d'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e47241'),
         watercraftType: 'Sailboat',
         ref: 'sailboats' } ],
    __v: 0 },
  { _id: ObjectId('5f7be0b37e2bdf5b19e4724e'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e47242'),
         watercraftType: 'Yatch',
         ref: 'yatches' } ],
    __v: 0 },
  { _id: ObjectId('5f7be0b37e2bdf5b19e4724f'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e47243'),
         watercraftType: 'Ship',
         ref: 'ships' } ],
    __v: 0 },
  { _id: ObjectId('5f7be0b37e2bdf5b19e47253'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e4724a'),
         watercraftType: 'Sailboat',
         ref: 'sailboats' },
       { _id: ObjectId('5f7be0b37e2bdf5b19e4724b'),
         watercraftType: 'Yatch',
         ref: 'yatches' },
       { _id: ObjectId('5f7be0b37e2bdf5b19e4724c'),
         watercraftType: 'Ship',
         ref: 'ships' } ],
    __v: 0 }
]

Stap 2 - $unwind zodat we kunnen herhalen met $loopup

In deze resultaatset is er een array van objecten met { _id: <ObjectId>, watercraftType: <ModelName> } om over de array te lopen en elk van deze objecten met hun respectieve verzamelingsrecord te verbinden, moeten we de array opsplitsen in afzonderlijke onafhankelijke records. De $unwind functie maakt een nieuwe gegevensset voor de volgende verzamelfase

  { '$unwind': '$watercraftContexts' },

$unwind Resultaat

Zoals je kunt zien $unwind maakt nu een record met een enkele watercraftContext we zijn nu ingesteld om de $lookup . te gebruiken

[ { _id: ObjectId('5f7be2231da37c5b5915bf9b'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf8f'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bf9c'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf90'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bf9d'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf91'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bfa1'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf98'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bfa1'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf99'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bfa1'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf9a'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0 } ]

Stap 4 $lookup - Voegt elk record uit de buitenlandse collectie samen

Het is belangrijk op te merken dat we $unwind . moeten voordat u $lookup calling aanroept voor elke verschillende collectie moeten we lid worden. Omdat we meerdere verzamelingen willen samenvoegen, moeten we het resultaat opslaan in een object dat door de verzameling is ingetoetst voor latere aggregatie.

  // Only performs $lookup on 'ships' collection
  { '$lookup':
     { from: 'ships',  // Collection Name - Note: repeat for each collection
       localField: 'watercraftContexts._id', // The field with id to link
       foreignField: '_id',  // The field on the foreign collection to match
       as: 'watercrafts.ships' // The path where to store the lookup result
     }
  }

Stap 5 - Herhaal de $unwind en $lookup voor de andere joins

Herhaal de bovenstaande stappen voor de extra joins en toets de verzamelingnaam in. Ik heb de geaggregeerde fasen gecombineerd om de herhaling te demonstreren.

  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'yatches',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.yatches' } },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'sailboats',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.sailboats' } }

Stap 4 &5 Resultaten

Als je goed kijkt, zie je dat een van de Captain records bestaan 3 keer met een ander watercraftType . $lookup zal alleen records retourneren die overeenkomen met een specifieke collectienaam. Bewaar ze daarom in een Object ingetoetst door collectionName

[
  { _id: ObjectId('5f7be7145320a65b942bb450'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb444'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches: [],
       sailboats:
        [ { _id: ObjectId('5f7be7145320a65b942bb444'),
            class: 'sail',
            name: 'Gone with the Wind',
            __v: 0 } ] } },
  { _id: ObjectId('5f7be7145320a65b942bb451'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb445'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches:
        [ { _id: ObjectId('5f7be7145320a65b942bb445'),
            class: 'yatch',
            name: 'Liquid Gold',
            __v: 0 } ],
       sailboats: [] } },
  { _id: ObjectId('5f7be7145320a65b942bb452'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb446'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     { ships:
        [ { _id: ObjectId('5f7be7145320a65b942bb446'),
            class: 'ship',
            name: 'Jenny',
            __v: 0 } ],
       yatches: [],
       sailboats: [] } },
  { _id: ObjectId('5f7be7145320a65b942bb456'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb44d'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches: [],
       sailboats:
        [ { _id: ObjectId('5f7be7145320a65b942bb44d'),
            class: 'sail',
            name: 'Swell Shredder',
            __v: 0 } ] } },
  { _id: ObjectId('5f7be7145320a65b942bb456'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb44e'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches:
        [ { _id: ObjectId('5f7be7145320a65b942bb44e'),
            class: 'yatch',
            name: 'Audrey',
            __v: 0 } ],
       sailboats: [] } },
  { _id: ObjectId('5f7be7145320a65b942bb456'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb44f'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     { ships:
        [ { _id: ObjectId('5f7be7145320a65b942bb44f'),
            class: 'ship',
            name: 'Jenny IV',
            __v: 0 } ],
       yatches: [],
       sailboats: [] } } ]

Stap 6 $project - Gebruik project om de Object Map van joins af te vlakken

We kunnen project gebruiken om alle bestaande gegevens te selecteren en de objectkaart van samenvoegingsresultaten af te vlakken tot een enkele array.

  { '$project':
     // keys with the value 'true' will be included
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts:            // Re-assigns value of watercrafts
        { '$setUnion':         // Accepts an array of arrays to flatten
           [
             '$watercrafts.ships',
             '$watercrafts.yatches',
             '$watercrafts.sailboats'
           ]
        }
     }
  }

$project Resultaat

De resultaten van het bovenstaande $project vervangt de watercrafts object met een afgevlakte array van watercrafts , maar het is belangrijk op te merken dat er nog steeds dubbele records zijn van Captain waar veel verschillende zoekopdrachten overeenkomen. We zullen ze in de volgende stap opnieuw samenvoegen.

[ { _id: ObjectId('5f7bea8d79dfe25bf3cb9695'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9689'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9689'),
         class: 'sail',
         name: 'Gone with the Wind',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb9696'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb968a'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb968a'),
         class: 'yatch',
         name: 'Liquid Gold',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb9697'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb968b'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb968b'),
         class: 'ship',
         name: 'Jenny',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9692'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9692'),
         class: 'sail',
         name: 'Swell Shredder',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9693'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9693'),
         class: 'yatch',
         name: 'Audrey',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9694'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9694'),
         class: 'ship',
         name: 'Jenny IV',
         __v: 0 } ] } ]

Stap 7 $unwind en $group

We $unwind zodat we nu alle watercrafts kunnen groeperen behorend tot dezelfde Captain . We moeten ook $mergeObjects . gebruiken om de aanvullende gegevens van de Captain tijdelijk op te slaan verzamelen onder een nieuwe tijdelijke variabele ter voorbereiding op de laatste fasen.

  { '$unwind': '$watercrafts' },
  { '$group':
     { _id: '$_id',
       data:
        { '$mergeObjects':
           { name: '$name',
             license: '$license',
             classes: '$classes',
             watercraftContexts: '$watercraftContexts',
             __v: '$__v' } },
       watercrafts: { '$push': '$watercrafts' } } }

$unwind en $group Resultaat

Nu komen we echt ergens. We hebben onze transformatie teruggebracht tot onze eerste 4 Captain s en onze joins afgeplat tot een enkele array.

[ { _id: ObjectId('5f7bed5e271dd95c306c25a4'),
    data:
     { name: 'CAPTAIN_SHIP',
       license: 'WC-3',
       classes: [ 'ship' ],
       watercraftContexts:
        { _id: ObjectId('5f7bed5e271dd95c306c2598'),
          watercraftType: 'Ship',
          ref: 'ships' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c2598'),
         class: 'ship',
         name: 'Jenny',
         __v: 0 } ] },
  { _id: ObjectId('5f7bed5e271dd95c306c25a8'),
    data:
     { name: 'CAPTAIN_SAIL_YATCH_SHIP',
       license: 'WC-7',
       classes: [ 'sail', 'yatch', 'ship' ],
       watercraftContexts:
        { _id: ObjectId('5f7bed5e271dd95c306c25a1'),
          watercraftType: 'Ship',
          ref: 'ships' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c259f'),
         class: 'sail',
         name: 'Swell Shredder',
         __v: 0 },
       { _id: ObjectId('5f7bed5e271dd95c306c25a0'),
         class: 'yatch',
         name: 'Audrey',
         __v: 0 },
       { _id: ObjectId('5f7bed5e271dd95c306c25a1'),
         class: 'ship',
         name: 'Jenny IV',
         __v: 0 } ] },
  { _id: ObjectId('5f7bed5e271dd95c306c25a2'),
    data:
     { name: 'CAPTAIN_SAIL',
       license: 'WC-1',
       classes: [ 'sail' ],
       watercraftContexts:
        { _id: Object('5f7bed5e271dd95c306c2596'),
          watercraftType: 'Sailboat',
          ref: 'sailboats' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c2596'),
         class: 'sail',
         name: 'Gone with the Wind',
         __v: 0 } ] },
  { _id: ObjectId('5f7bed5e271dd95c306c25a3'),
    data:
     { name: 'CAPTAIN_YATCH',
       license: 'WC-2',
       classes: [ 'yatch' ],
       watercraftContexts:
        { _id: ObjectId('5f7bed5e271dd95c306c2597'),
          watercraftType: 'Yatch',
          ref: 'yatches' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c2597'),
         class: 'yatch',
         name: 'Liquid Gold',
         __v: 0 } ] } ]

Stap 8 $replaceRoot en $project

Het enige dat we nog moeten doen, is onze data samenvoegen in de root van elk record en verwijder de tijdelijke variabele data

  // Merges 'data' into the root of each record
  { '$replaceRoot': { newRoot: { '$mergeObjects': [ '$data', '$$ROOT' ] } } },
  // Use $project to remove data (include only the fields we want)
  { '$project':
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts: true } 
  }

$replaceRoot &$project Resultaat

Nu hebben we het resultaat dat we hebben opgesteld voor... Een Captain met een reeks van gemengde geassocieerde typen watercrafts

[ 
  { name: 'CAPTAIN_SAIL_YATCH_SHIP',
    license: 'WC-7',
    classes: [ 'sail', 'yatch', 'ship' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755ea6'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ead'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755ea4'),
         class: 'sail',
         name: 'Swell Shredder',
         __v: 0 },
       { _id: ObjectId('5f7bf3b3680b375ca1755ea5'),
         class: 'yatch',
         name: 'Audrey',
         __v: 0 },
       { _id: ObjectId('5f7bf3b3680b375ca1755ea6'),
         class: 'ship',
         name: 'Jenny IV',
         __v: 0 } ] },
  { name: 'CAPTAIN_SAIL',
    license: 'WC-1',
    classes: [ 'sail' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755e9b'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ea7'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755e9b'),
         class: 'sail',
         name: 'Gone with the Wind',
         __v: 0 } ] },
  { name: 'CAPTAIN_YATCH',
    license: 'WC-2',
    classes: [ 'yatch' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755e9c'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ea8'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755e9c'),
         class: 'yatch',
         name: 'Liquid Gold',
         __v: 0 } ] },
  { name: 'CAPTAIN_SHIP',
    license: 'WC-3',
    classes: [ 'ship' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755e9d'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ea9'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755e9d'),
         class: 'ship',
         name: 'Jenny',
         __v: 0 } ] } ]

En daar heb je het ... duurde slechts 2 dagen om dit uit te zoeken. Ik hoop dat het je wat tijd bespaart als je een vergelijkbare geaggregeerde associatie probeert. Veel plezier met coderen!

Definitieve pijplijn

[ 
  { '$match':
     { _id:
        { '$in':
           [ ObjectId('5f7bf3b3680b375ca1755ea9'),
             ObjectId('5f7bf3b3680b375ca1755ea7'),
             ObjectId('5f7bf3b3680b375ca1755ea8'),
             ObjectId('5f7bf3b3680b375ca1755ead')
           ]
        }
     }
  },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'ships',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.ships' } },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'yatches',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.yatches' } },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'sailboats',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.sailboats' } },
  { '$project':
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts:
        { '$setUnion':
           [ '$watercrafts.ships',
             '$watercrafts.yatches',
             '$watercrafts.sailboats' ] } } },
  { '$unwind': '$watercrafts' },
  { '$group':
     { _id: '$_id',
       data:
        { '$mergeObjects':
           { name: '$name',
             license: '$license',
             classes: '$classes',
             watercraftContexts: '$watercraftContexts',
             __v: '$__v' } },
       watercrafts: { '$push': '$watercrafts' } } },
  { '$replaceRoot': { newRoot: { '$mergeObjects': [ '$data', '$$ROOT' ] } } },
  { '$project':
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts: true } }
]