Nl/the real streaming log files

[NatLeung96]

Here is an actual (draft) PR for log streaming.

It "works" however the subscription appears to be sending duplicate data which is why the log readout repeats a lot. I'm at complete odds and would like to see if some other eyes might have some idea.

As I mentioned in the sprint planning meeting, I moved the log streaming, log inspecting, and plotting components into a sub-component with the intention that they would all use fragments from the shared query. However, I seem to be having a weird issue with vscode crashing whenever I try to read the Plot.tsx file (has anyone seen that?). The plot seems to work as is for now so perhaps we can leave that for the refactor later on? I can then start working on the other components I need to do for Monday.

[yousefmoazzam]

I've not gone through the PR yet, but will try to make a few comments on the questions you asked.

Regarding the duplicated data from the subscription, the websocket messages in the devtools network tab indicates that there is only two messages of information before the websocket connection is closed, so I think there is no duplicated data at the network level (ie, the workflows platform is sending us the correct amount of data):

After turning off react's strict mode locally (since it makes things more confusing by purposely running some things twice for the sake of catching certain programmer errors - this makes there be only 1 copy of the first message, and 2 copies of the second message), when inspecting the writes to the Apollo cache, one can see that there are three writes to the cache: the first two are the new lines that come from the two messages, but then the third and last write is a duplicate of the second message:

A cursory look around online suggests that care needs to be taken to ensure that objects received from subscription messages need to be uniquely identifiable in some manner before working with it in the application code, or to manually check for uniqueness before appending to whatever data structure is desired.

I don't know enough about Apollo and Relay to say if this is a quirk unique to Apollo, a cursory look in the Relay docs say the following though:

Note that the event stream can be completely arbitrary, and can have no relation to the fields selected. In other words, there is no guarantee that the values selected in a subscription will have changed from notification to notification.

which makes me think this sort of thing is perhaps not Apollo specific, but possibly more to do with GraphQL subscriptions in general?

If so, it may be worth poking around online to see how people deal with checking if data is unique or not before pushing it into the local data structure.

As a side note, if it's indeed the case that this is something to do with GraphQL subscriptions in general, but that this happens in less cases with Relay than Apollo, then it may be that Relay is doing things for the user relating to caching, which wouldn't be the first time we came across such an idea (namely, Relay handling fragments and generating unique cache IDs for the user transparently if no ID field on an object exist, whereas in Apollo it was necessary for us to generate a unique cache ID manually).

Regarding the question about VSCode, I don't use it so I can't comment on the crashing of it when opening the Plot.tsx file; the only thing I can report is that the file opens fine in neovim, so I'd suggest seeing if VSCode produces any logs for debugging.

[NatLeung96]

Not sure what I did but vscode seems to be behaving again so I've updated Plot to use fragments

[NatLeung96]

There appears to be two ways of performing subscriptions in Apollo. I switched to the other method and that seemed to solve the problem.