Read a dataset from a set of files
Read files into a dataset, optionally processing them in parallel.
read_files(files, reader, ..., parallel_files = 1, parallel_interleave = 1, num_shards = NULL, shard_index = NULL)
List of filenames or glob pattern for files (e.g. "*.csv")
Additional arguments to pass to
An integer, number of files to process in parallel
An integer, number of consecutive records to produce from each file before cycling to another file.
An integer representing the number of shards operating in parallel.
An integer, representing the worker index. Shared indexes are 0 based so for e.g. 8 shards valid indexes would be 0-7.