Breast cancer is a pervasive malignancy worldwide, claiming numerous lives each year. Early and accurate detection of cancer through efficient and cost-effective methods is crucial in minimizing its impact. While mammary ductal images have been foundational in breast cancer diagnosis, there is still a need to improve their accuracy and efficiency. Mammography, particularly when utilizing mammary ductal images, plays a vital role in breast cancer screening by offering a non-invasive approach to identify potential abnormalities. Early detection through precise image analysis not only helps halt the disease's progression but also significantly enhances patient survival rates. However, the interpretation of mammograms requires specialized expertise, and the limited availability of skilled radiologists highlights the necessity for a reliable and automated solution. Our project was initiated to address this challenge by integrating deep learning into mammographic image analysis. We have tailored the widely acclaimed U-Net architecture to our specific goal: detailed and multi-class segmentation of mammary ductal images. Key Innovations: Multi-class Pretraining: We leverage extensive multi-class pretraining to equip the model with a comprehensive understanding of diverse image attributes and abnormalities. This enables more nuanced and accurate image segmentation. SAM Prompt: By incorporating the Segment Anything Model (SAM) prompt engineering, we enhance the model's convergence rate. Additionally, it significantly improves the model's robustness and precision, enabling the detection of subtle structures and anomalies within the images.