If you're looking for a low key spot out in the middle of nowhere, I reckon you head to Little Desert NP. The park is located about three and a half hours west of the Grampians. We stopped here on New Years Eve to break up the long haul to Kangaroo Island.
We had the entire campsite to ourselves. Also, since the park is literally in the middle of nowhere we had the most amazing display of stars that we had seen while in Australia. We even caught a glimpse of the Aurora Australis.
Apparently, Little Desert is Bird Watching haven. As we were packing up our car we spotted an Emu and two chicks, which was the icing on the cake. If you're passing by this small park and want a night of solitude, we highly recommend it. Bring firewood! There are sweet little fire pits in each campsite.