I am in the UK too! You definitely should be wearing a swim cap in that case, any decently managed pool here should be asking customers to water only shower before getting in the pool to remove product residues and body fat which contaminate the water. This is taught in our pool hygiene certification, which anyone doing the pool tests (every few hours) should hold.
You might try super lightweight silicone free conditioners often used for cowashing but as your main conditioner: Inecto coconut conditioner or many of the pharmacy/ supermarket own brand (the cheapest Basics/ Value type) are options. My hair is 'squeaky' clean after a co-wash with Inecto so definitely no residue or build up, it's far too light for me as a conditioner but it gets very good reviews on Amazon as such. I buy it at Bodycare or Home Bargains for ~£1.50.
Inecto Pure Coconut Conditioner 500ml: Amazon.co.uk: Beauty
More expensive is CURLS Milkshake which is available at the British Curlies Curly Emporium and can be used as a light milky conditioner and/ or a light styler, you could water it down if still too heavy. They also sell Komaza Care which have some lovely products at pH 4.5 some of which are very lightweight. Options which you might read reviews on (or contact Komaza for advice, I hear they are very helpful) include the coconut hair milk, Matani repair spray (protein and ceramides), coconut curl spray.
You don't have to use a leave in if you don't want to, but not using conditioner to either wash or (rinse out) condition means you are completely missing out on a range of protective benefits, any combing or detangling you do is more likely to be damaging. Needing something so silicone heavy to have hair that looks and feels healthy suggests to me something else in your routine or lifestyle is amiss. Are you consistently meeting or exceeding ALL our government's healthy eating and living guidelines? Have you seen your family doctor about this? Are you tying your hair up a lot, a very restless sleeping, is your hair getting rubbed with a headrest in your car or workplace?
Naturallythinking have a sulphate free shampoo base, it's pH 7 tho so you'd want to reset the pH afterwards or mix with some dilute vinegar right before use. You can use as is, or add some coconut oil to try the Movie Star Method (see Long Hair Community); it definitely removes silicones. Also Soapkitchenonline is often mentioned on forums but I don't know the pH of that one. Either one if you don't like the product you could use it up as shower gel, the formula would work fine.
Shampoo SLS free Extra Mild - Naturallythinking Pure Aromatherapy and Spa
Komaza Care have Moja shampoo which is pH 4.5. Not a base but some lovely ingredients, none of which should build up. It *might* remove silicones since they do have silicones in one of their products, you'd be best to ask them.