Scale doesn't matter since the sphere is just generating directional light rays from a theoretical infinite position. The sphere is little more than a gizmo to show rotation. I don't even think your objects need to be inside the sphere to be lit by it. This is different however from using a normal sphere like an IBL object using a radiance map, which can be scaled and moved if that's what you need to do.
well I just wanted to mirror the image on the IBL sphere, so I have add a minus to the scale, but it didn't work on the rendered image. (but only on the rendered image). so finally I grabbed the lighting image, and mirrored it in a photo program. I think in this case, it had to be in the middle of the IBL sphere. here is the final render.