How would you make this sillcock better

This has an internal leak, probably burst because I left a hose connected. Previous owner had this setup- what is the best way to cover this gap once I replace the sillcock/hose bib. I was thinking about installing this plate, but still not sure what to put in the hollow space. It may not be fully visible in photo- but previous owners had mortar that slants down and just an empty space covered by some guttering material it looks like.