Multiple people as a "client"-toggle on and off as needed for builders.

I would like to be able to add a builder or contractor as a client as well as the homeowner. Sometimes I need to send an invoice to just the builder but I don’t want the client to be able to see it. I was hoping there might be a way to have 2 "clients" on one project and be able to toggle on and off permissions for things like the invoices, etc. I would love to be able to say who gets to see what all on the same project.