I may be wrong, but might it be you are trying to broadcast at too high a frame rate and too high a resolution from an underpowered computer?
I don't use Streaming Video for astronomy, but do so via Youtube to broadcast live cricket. Here are my thoughts...
I embrace an end to end 4K UHD system for EAA astronomy using my (original) Atik Horizon. Here, I use 802.11ac WiFi and Windows Remote Desktop to remote control everything from indoors. It took a long development time to eliminate all the bottlenecks in this system. However, everything technical was in my control, and I could upgrade to any necessary faster components, unlike with YouTube and Internet. That, and your computer might be a bottleneck.
Hence, by contrast, I must limit the cricket video broadcast to 1080p and around 19fps and that is when using my broadcast camera's H265 video compression. It is highly likely (IMHO) you are generating too much data from the Infinity with no benefit from software compression. I also found that the streaming video process also chokes on anything less than an 8th generation i7 processor with 16Gb RAM even at the lower resolution. What processor and RAM do you have?