Soil for sod?

We had an above ground pool and now we have about an inch or two of sand and a lot of gravel. Is it Ok to rake out the the gravel and mix it with the sand and topsoil or should the gravel be removed first?