The system behaves in a strange way. I owned some guys two weeks ago. In average (kills divided by people in the enemy team) I killed them 11.5 times each. My Favorite Victim didn't change and their Worst enemy didn't either. Then I killed one guy seven times in a row with arty on Kubra Dam. He was AFK and an easy victim for my barrage. He is now my Favorite Victim with 9 kills but I'm not his Worst Enemy, who killed him 7 times.
Maybe you have to kill someone more times than your FV in one round and kill him more times in a row (e.g. without killing someone else than him) in the same round.
Fav Victim:
A kills B 10 times in a round, A kills B 5 times in a row without killing another player => A's FV: B
A kills C 16 times in a round, A kills C 4 times in a row without killing another player => A's FV: B
A kills D 9 times in a round, A kills D 6 times in a row without killing another player => A's FV: B
A kills E 11 times in a round, A kills E 6 times in a row without killing another player => A's FV: E
vive versa Worst Enemy:
A kills B 10 times in a round, B dies by A 5 times in a row without getting by killed by another player => B's WE: A
C kills B 16 times in a round, B dies by C 4 times in a row without getting by killed by another player => B's WE: A
D kills B 9 times in a round, B dies by D 6 times in a row without getting by killed by another player => B's WE: A
E kills B 11 times in a round, B dies by E 6 times in a row without getting by killed by another player => B's WE: D