Pakistanis have made it difficult for Nawaz Sharif to come out of his home