How do I turn automatic payments on or off with a Shopify Collective retailer?

You can turn automatic payments off with a specific retailer directly from Retailers > Connected in your Collective: Supplier sales channel; the retailer is notified by email, and it's then your responsibility to agree on a manual payment method with them. Turning payments back on can only be done by the retailer, so you'd need to ask them directly.