The Congress has said it would back Vijay's TVK, the newly formed party which made a blockbuster debut in the Tamil Nadu assembly elections but failed to get the majority.