Allay spawn naturally in Pillager outposts and woodland mansions. Pillager outposts generate in any village-generating biome (such as plains, desert, etc.) while woodland mansions spawn only in the dark forest biome.
There is no specific item that will bind an allay to a player. Because its function and purpose is to find lost items, all you need to do is give the allay an item you want it to find!
The allay will follow you, seeking out dropped versions of the same item and delivering them to you. It follows a player from up to 64 blocks away even if it can't see them, and it seeks items up to 32 blocks away from the player.
After delivering items, there is a 3 second delay until it can seek items again. The allay has a single inventory slot to hold a stack of items (usually 64 items) in addition to the item it got from the player. So, this will make having an allay find unstackable items, like potions, tedious.
To stop an allay from following you, take its item back by using an empty hand on it. You can use a lead and name tag on allays!