Abstract Aims Although platinum-based combination chemotherapies are commonly used for unfavourable subsets of cancer of unknown primary (CUP), the prognosis remains poor. Several studies have suggested that gene expression profiling or immunohistochemistry was useful for the prediction of primary sites in CUP, and site-specific therapy based on predicted primary sites might improve overall outcomes. In Japan, to identify primary sites, immunohistochemical tests were commonly used for CUP in clinical practice. However, it is unclear whether site-specific therapy based on predicted primary sites by pathological examination contributes survival benefit for unfavourable CUP subsets. Patients and methods In this study, 122 patients with unfavourable subsets of CUP were retrospectively reviewed. Ninety patients assigned to cohort A after July 2012 had received chemotherapy according to predicted primary sites; 32 patients assigned to cohort B before June 2012 had received platinum-based empiric chemotherapy. Results In cohort A, 56 patients (62.2%) with predicted primary sites by pathological examination received site-specific therapy; 34 patients (37.8%) with unpredictable primary sites received platinum-based empiric chemotherapy, the same as cohort B. The median overall survival was 20.3 months in patients with predictable primary sites in cohort A and 10.7 months in those of cohort B, with a significant difference between these cohorts (P = 0.03, adjusted hazard ratio = 0.57, 95% confidence interval 0.34–0.94). Conclusion Site-specific therapy based on predicted primary sites by pathological examination could improve prognosis in patients with an unfavourable subset of CUP.