It all depends. It depends on the speed, type & quality of network (e.g. is it micro-segmented or shared, how good are your switches), it depends on the size & frequency of the packets, the number of broadcasting clients, etc. If you're running a routed network i.e. multiple subnets, how (if at all) are you intending to handle broadcasts to the non-native subnets? How will the routers handle this? It depends on the capability of your end devices too, they'll need to process every UDP broadcast frame - at high rates this can slow down low-end machines quite considerably. Don't let this put you off though, if you've ever done a network trace then, unless you're on a microsegmented LAN, you'll probably see quite a bit of background broadcast traffic anyway and it all ticks along happily.
It might be worth reading up on multicast groups and seeing if that might be an option for your application as there are ways, with various types of network equipment, that you can configure your network to handle multicast more efficiently that straight UDP broadcasts.