In this paper we describe results of an experimental study where we checked the impact of various difficulty factors in imbalanced data sets on the performance of selected classifiers applied alone or combined with several preprocessing methods. In the study we used artificial data sets in order to systematically check factors such as dimensionality, class imbalance ratio or distribution of specific types of examples (safe, borderline, rare and outliers) in the minority class. The results revealed that the latter factor was the most critical one and it exacerbated other factors (in particular class imbalance). The best classification performance was demonstrated by non-symbolic classifiers, particular by $k$-NN classifiers (with 1 or 3 neighbors -- 1NN and 3NN, respectively) and by SVM. Moreover, they benefited from different preprocessing methods -- SVM and 1NN worked best with undersampling, while oversampling was more beneficial for 3NN.