The following are listed on the Norfolk Pubs (8) website but with little information

Antelope, Duke St/Quebec St (possibly became the Half Moon)
Duke of York, Duke St/Quebec St (possibly became Dukes Head)
Gun - Pigots 1830 - Charles Secker, Gun, Church St.
Halfway House
Prince of Wales
Recruiting Sergeant
Refreshment Rooms - Railway Station (not a pub in my opinion)
Queens Hotel - Kelly's 1908 lists George Wake, Veterinary Surgeon
and proprietor of Queens Temperance Hotel!

Queens Temperance Hotel

The 1757 Boston & Puddy map (1) shows a demolished Inn on Guild Hall Row

Duke Street / Quebec Street "Quebec house, the seat of Col. William Earle Gascoyne Lytton Bulwer J.P. High Sheriff of Norfolk, 1883, is situate a mile north-west from the town, is Gothic, and derives its name from having been built at the time of the capture of Quebec; it stands in an extensive lawn, which is well timbered, and is partly surrounded by plantations." (9)

Did Duke Street become Quebec Street when Quebec Hall was built?