The most straightforward way to do this would be with track mattes.
You would create an image in photoshop, OR I quite frequently just use the title designer fr this as it works fine... The image should be black where you don't want video and white where you do.
I would create a separate matte for each video "window".
Then you can take the matte and do this two ways.
1. just set the video you want to use on the timeline with the matte somewhere above and select "track matte" for the source clip, selecting the track the matte is placed on as the source for the matte and also select "luma" as the matte mechanism.
When you do it this way, if you alter the size or position of the source clip, the matte will follow.
2. You can set the matte and you would place the source clip on another sequence and alter its position or scale, then nest that sequance as the source clip and place the track matte on that nested sequence instead of the source clip as in #1.