You only have intuition or their direct actions with you. Even then you eventually have to ask. There is a degree of trust involved.