Module ojalgo
Package org.ojalgo.netio

Class SegmentedFile

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    public class SegmentedFile
extends java.lang.Object
implements java.lang.AutoCloseable
    Divides a large file in segments and then reads those (in parallel) using memory mapped files (memory mapped file segments). Basic usage:
    1. Call newBuilder(File) to get a SegmentedFile.Builder or of(File) to get a SegmentedFile directly with the default settings.
    2. Then for each of the segments obtained from segments(): Call newTextLineReader(Segment) and using that read all the lines.
    Multiple threads can be used to read the segments in parallel. Each thread just works on one segment at a time, and there's a new TextLineReader for each segment.

    • Field Detail

      • myFileChannel

        private final java.nio.channels.FileChannel myFileChannel

      • myRandomAccessFile

        private final java.io.RandomAccessFile myRandomAccessFile

    • Constructor Detail

    • Method Detail

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable

      • newSequencedFactory

        public <T> java.util.function.Supplier<FromFileReader<T>> newSequencedFactory​(java.util.function.Function<SegmentedFile.Segment,​FromFileReader<T>> factory)
        Each reader instantiated by this factory will read from the segments in sequence, until all of them are done. The idea is that you can create multiple readers and have them work in parallel (each segment will only be read once by one of the readers). If you only instantiate 1 reader, then maybe you shouldn't have created file segments in the first place.

      • newTextLineReader

        public FromFileReader<java.lang.String> newTextLineReader()

      • newTextLineReader

        public TextLineReader newTextLineReader​(SegmentedFile.Segment segment)
        Call this once for each file segment, and use the returned TextLineReader to read the file segment. The TextLineReader is not thread safe, and should only be used by a single thread.

        The segment is a range of bytes in the file. The TextLineReader will read the bytes in the range [offset, offset + size). The segments are obtained from segments(). The segments are sorted in ascending order by offset.