When using GridSearch, don't most people start with a coarse (few steps for each parameter) grid and then add more fidelity? I certainly would start with a 100x100 parameter space, if anything my first pass I might do 2 or 3 steps per parameter to get a quick result first.


If you haven't come across approach then I think you've missed a crucial step, which can provide an alternative approach to using the very same GridSearch which provides more flexibility then the HalveSearch.

Instead of iterating over small increments of the hyperparameters use coarse increments. For example, instead cover a range of 0 to 1 in a 100 steps, use 5 steps.

This coarse initial exploration will run quicker and may indicate which parameters provider materially different results. These parameters can then refined with greater fidelity over other parameters which appear to be less important.