It is well-known that the chemical potential which drives the vapor-liquid-solid growth of semiconductor nanowires is strongly affected by the liquid phase composition. Here, we investigate theoretically how the droplet composition influences the nucleation of Au-catalyzed GeSn nanowires on Ge(111) and Si(111) substrates. We compare the chemical potentials in an Au-Ge-Sn catalyst droplet before and after adding Ga and/or Si atoms. It is found that the presence of these atoms enhances the nucleation rate of nanowires on both substrates. Theoretical results are compared to experimental data on GeSn nanowires grown in a hot-wall reduced pressure chemical vapor deposition reactor. It is shown that the intentional addition of Ga in the de-wetting step improves the uniformity of the nanowire dimensions and yields higher density of nanowires over Ge(111) substrates. The nanowire growth on Si(111) substrate occurs only when Ga and/or Si are added to Au droplets. These results show that controlling the composition of the catalyst droplet is crucial for improving the quality of GeSn nanowires.