Prime Minister Narendra Modi had spent considerable time campaigning in West Bengal ahead of the assembly election