| Name : R-ff
| |
| Version : 4.5.0
| Vendor : obs://build_opensuse_org/devel:languages:R
|
| Release : lp154.1.1
| Date : 2024-09-23 10:55:01
|
| Group : Development/Libraries/Other
| Source RPM : R-ff-4.5.0-lp154.1.1.src.rpm
|
| Size : 1.53 MB
| |
| Packager : https://www_suse_com/
| |
| Summary : Memory-efficient storage of large data on disk and fast access functions
|
Description :
The ff package provides data structures that are stored on disk but
behave (almost) as if they were in RAM by transparently mapping only a
section (pagesize) in main memory - the effective virtual memory
consumption per ff object. ff supports R\'s standard atomic data types
\'double\', \'logical\', \'raw\' and \'integer\' and non-standard atomic types
boolean (1 bit), quad (2 bit unsigned), nibble (4 bit unsigned), byte
(1 byte signed with NAs), ubyte (1 byte unsigned), short (2 byte signed
with NAs), ushort (2 byte unsigned), single (4 byte float with NAs).
For example \'quad\' allows efficient storage of genomic data as an
\'A\',\'T\',\'G\',\'C\' factor. The unsigned types support \'circular\'
arithmetic. There is also support for close-to-atomic types \'factor\',
\'ordered\', \'POSIXct\', \'Date\' and custom close-to-atomic types. ff not
only has native C-support for vectors, matrices and arrays with
flexible dimorder (major column-order, major row-order and
generalizations for arrays). There is also a ffdf class not unlike
data.frames and import/export filters for csv files. ff objects store
raw data in binary flat files in native encoding, and complement this
with metadata stored in R as physical and virtual attributes. ff
objects have well-defined hybrid copying semantics, which gives rise to
certain performance improvements through virtualization. ff objects can
be stored and reopened across R sessions. ff files can be shared by
multiple ff R objects (using different data en/de-coding schemes) in
the same process or from multiple R processes to exploit parallelism. A
wide choice of finalizer options allows to work with \'permanent\' files
as well as creating/removing \'temporary\' ff files completely
transparent to the user. On certain OS/Filesystem combinations,
creating the ff files works without notable delay thanks to using
sparse file allocation. Several access optimization techniques such as
Hybrid Index Preprocessing and Virtualization are implemented to
achieve good performance even with large datasets, for example virtual
matrix transpose without touching a single byte on disk. Further, to
reduce disk I/O, \'logicals\' and non-standard data types get stored
native and compact on binary flat files i.e. logicals take up exactly 2
bits to represent TRUE, FALSE and NA. Beyond basic access functions,
the ff package also provides compatibility functions that facilitate
writing code for ff and ram objects and support for batch processing on
ff objects (e.g. as.ram, as.ff, ffapply). ff interfaces closely with
functionality from package \'bit\': chunked looping, fast bit operations
and coercions between different objects that can store subscript
information (\'bit\', \'bitwhich\', ff \'boolean\', ri range index, hi hybrid
index). This allows to work interactively with selections of large
datasets and quickly modify selection criteria. Further
high-performance enhancements can be made available upon request.
|
RPM found in directory: /packages/linux-pbone/ftp5.gwdg.de/pub/opensuse/repositories/devel:/languages:/R:/autoCRAN/15.4/x86_64 |
