Can a contractor use my mortgage draws for another house that he is working on?

I think my contractor has been using my bank draws towards other houses he is working on. Would like to know if i have legal action against him