Artificial Intelligence (AI) has provided user experience (UX) designers with a richer toolset. To use technologies such as Machine Learning (ML) that can expand their creative capacity to design intelligent services. ML has the capability to enhance the user experience, for example, by improving efficiency, personalization, and context-aware adaptation. However, research suggests ML as a challenging design material in UX practice, such as difficulties in comprehending data dependencies when prototyping, or the lack of tools and methods for evaluating adaptive user experiences. Previous research indicates that lack of knowledge transfer into the UX design practice may hamper innovative potential. This work aims to provide new insights on how designers think about – and experience – design for AI-powered services. It is important to make ML-powered services beneficial and sustainable for end-users, organizations, and society. Therefore, we explore UX designers’ reflections and experiences of using ML in a design context. We have performed nine deep explorative interviews with professional designers that work with ML. The respondents have different backgrounds, seniority, and work in different sectors. The collected interview material was qualitatively analyzed and resulted in five conceptual themes for how UX designers experience the design context surrounding AI-powered services: 1) Absence of competence, 2) Lack of incentive for competence development, 3) Challenges in articulating design criteria, 4) Mature vs. Immature clients, and 5) Lack of support for ethical concerns. We provide implications for how these themes affect the design context and practice. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.