As a person who has played this game, I think the best way to ask this is the simple question.
Do you desire DPS or a Elemental Support?
Now if you want my opinion,
Kazuha’s main role is to do damage but to also give support as he gives Elemental damage bonus based off his elemental mastery. Essentially, he can give any elemental damage based character a well deserved boost and due to the fact that he is an anemo character, it also means he can use the anemo set which is one of the best sets in the game.
Klee is one the best main dps, she’s not as constant as Ganyu and Diluc but true Klee mastery makes you really hit bosses and any opponent extremely hard. Like Kazuha, she’s lucky to have one of the more better sets, this of course being the Crimson Witch set.
Personally, I think Kazuha would be the better choice if you already have a dps. Especially since both elemental boost and debuff is the first of its kind. Also because characters like Yanfei also exist, who is like a weaker Klee with Charged attack spam. but it’s your choice
Edit: I apologize for my slowness in making this answer, got sidetracked in Genshin by Eula.
Edit 2: For anyone confused about “true Klee Mastery” Klee can be really powerful with animation canceling and understanding how to make her move without being slow.