When I was replacing my rear speakers, I gained enough access by removing the following:

1. The cover that runs along the bottom of the hatch opening, where the hatch door locks into.

2. The tie down loops on each side of the rear hatch.

3. All of the push-in fasteners that you see when you remove those small triangular shaped covers for the storage bins on each side of the hatch.

4. Remove the hard plastic tray in the middle of the hatch and the storage tray underneath to make it easier to fold the stiffened carpet that should now be loose.

I'm doing this from memory so there might be some screws I missed but I think I've covered everthing. Once you have done all the above steps, the side panels should be able to be popped out. When you are pulling them out, there will be some resistance as there are still some retaing clips that hold them on. I used a small piece of wood to hold them out while working behind them.