While urban sound management often focuses on sound as a nuisance, soundscape research suggests that proactive design approaches involving sound art installations can enhance public space experience. Nevertheless, there is no consensus on a methodology to inform the composition of sound installations through soundscape evaluation, and little research on the effect of composition strategies on soundscape evaluation. The present study is part of a research-creation collaboration around the design of a permanent sound installation in an urban public space in Paris (Niches Acoustiques by Nadine Schütz). We report on a laboratory study involving the evaluation of composition sketches prior to the deployment of the installation on-site. Participants familiar with the public space (N = 20) were exposed to Higher-Order Ambisonics recordings of the site, to which compositions of the sound installation pertaining to different composition strategies were added using a soundscape simulation tool. We found three principal components relevant for evaluating and comparing sound installation sketches: pleasantness, familiarity and variety. Further, all composition sketches had a significant effect on the soundscape's familiarity and variety, and the effect of the compositions on these two components was stronger when composition strategies involved abstract sounds (sounds which were not clearly identifiable).