See https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#input-sources
Questions:
- Does it read the whole file, and basically allows working with static sources in conjunction with streaming sources, or does it only process the latest/unseen data? I.e. is this a way of persisting/loading streaming file data, a way to integrate batch data, or both?
See https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#input-sources
Questions: