The emergence of data-driven science has opened up new avenues for understanding the thermophysical properties of materials. For decades, alloys are known to possess very low thermal conductivity, but the extreme thermal conductivity that can be achieved by alloying has never been identified. In this work, we combine the Bayesian optimization with a high throughput thermal conductivity calculation to search for the lowest thermal conductivity atomic configuration of SiGe alloy. It is found layered structures are most beneficial for reducing the thermal conductivity among all atomic configurations, which is attributed to the strong branch-folding effect. Furthermore, the roles of interface roughness and layer thicknesses in producing the lowest thermal conductivity are investigated. Through another comprehensive search using Bayesian optimization, the layered structure with smooth interfaces and optimized layer thickness arrangement is eventually identified as the optimal structure with the lowest thermal conductivity.