Knotweed does not respect fences. The root systems are capable of traveling 20m horizontally — so if you have it, your neighbour probably does too.
Knotweed is serious stuff. In the UK some homeowners have been denied property insurance or mortgages on properties with knotweed. Here in BC, this hasn’t happened yet. The BC Weed Control Act does require any land owner or occupier to control knotweed on their property.