pocketutils.tools.filesys_tools

Module Contents

class pocketutils.tools.filesys_tools.PathInfo

Info about an extant or nonexistent path as it was at some time. Use this to avoid making repeated filesystem calls (e.g. .is_dir()): None of the properties defined here make OS calls.

source

The original path used for lookup; may be a symlink

resolved

The fully resolved path, or None if it does not exist

as_of

A datetime immediately before the system calls (system timezone)

real_stat

os.stat_result, or None if the path does not exist

os.stat_result, or None if the path is not a symlink

has_access

Path exists and has the ‘a’ flag set

has_read

Path exists and has the ‘r’ flag set

has_write

Path exists and has the ‘w’ flag set

All the additional properties refer to the resolved path, except for is_symlink(), is_valid_symlink(), and is_broken_symlink().

property mod_or_create_dt → Optional[datetime.datetime]

Returns the modification or access datetime. Uses whichever is available: creation on Windows and modification on Unix-like.

property mod_dt → Optional[datetime.datetime]

Returns the modification datetime, if known. Returns None on Windows or if the path does not exist.

property create_dt → Optional[datetime.datetime]

Returns the creation datetime, if known. Returns None on Unix-like systems or if the path does not exist.

property access_dt → Optional[datetime.datetime]

Returns the access datetime. Should never return None if the path exists, but not guaranteed.

property exists → bool

Returns whether the resolved path exists.

class pocketutils.tools.filesys_tools.FilesysTools

Tools for file/directory creation, etc.

Caution

Some functions may be insecure.

classmethod read_compressed_text(path: pocketutils.core.input_output.PathLike) → str

Reads text from a text file, optionally gzipped or bz2-ed. Recognized suffixes for compression are .gz, .gzip, .bz2, and .bzip2.

classmethod write_compressed_text(txt: str, path: pocketutils.core.input_output.PathLike, *, mkdirs: bool = False) → None

Writes text to a text file, optionally gzipped or bz2-ed. Recognized suffixes for compression are .gz, .gzip, .bz2, and .bzip2.

classmethod prep_dir(path: pocketutils.core.input_output.PathLike, *, exist_ok: bool = True) → bool

Prepares a directory by making it if it doesn’t exist. If exist_ok is False, calls logger.warning if path already exists

classmethod prep_file(path: pocketutils.core.input_output.PathLike, *, exist_ok: bool = True) → None

Prepares a file path by making its parent directory. Same as pathlib.Path.mkdir but makes sure path is a file if it exists.

classmethod dump_error(e: Optional[BaseException], path: Union[None, pocketutils.core.input_output.PathLike, datetime.datetime] = None) → pathlib.Path

Writes a .json file containing the error message, stack trace, and sys info. System info is from get_env_info().

classmethod delete_surefire(path: pocketutils.core.input_output.PathLike) → Optional[Exception]

Deletes files or directories cross-platform, but working around multiple issues in Windows.

Returns

None, or an Exception for minor warnings

Raises

IOError – If it can’t delete

classmethod trash(path: pocketutils.core.input_output.PathLike, trash_dir: Optional[pocketutils.core.input_output.PathLike] = None) → None

Trash a file or directory.

Parameters

  • path – The path to move to the trash

  • trash_dir – If None, uses pocketutils.tools.path_tools.PathTools.guess_trash()

classmethod try_cleanup(path: pathlib.Path, *, bound: Type[Exception] = PermissionError) → None

Try to delete a file (probably temp file), if it exists, and log any PermissionError.

classmethod read_lines_file(path: pocketutils.core.input_output.PathLike, *, ignore_comments: bool = False) → Sequence[str]

Returns a list of lines in the file. Optionally skips lines starting with # or that only contain whitespace.

classmethod read_properties_file(path: pocketutils.core.input_output.PathLike) → Mapping[str, str]

Reads a .properties file. A list of lines with key=value pairs (with an equals sign). Lines beginning with # are ignored. Each line must contain exactly 1 equals sign.

Caution

The escaping is not compliant with the standard

Parameters

path – Read the file at this local path

Returns

A dict mapping keys to values, both with surrounding whitespace stripped

classmethod write_properties_file(properties: Mapping[Any, Any], path: Union[str, pathlib.PurePath], mode: str = 'o') → None

Writes a .properties file.

Caution

The escaping is not compliant with the standard

classmethod read_any(path: pocketutils.core.input_output.PathLike) → Union[str, bytes, Sequence[str], pandas.DataFrame, Sequence[int], Sequence[float], Sequence[str], Mapping[str, str]]

Reads a variety of simple formats based on filename extension. Includes ‘.txt’, ‘csv’, .xml’, ‘.properties’, ‘.json’. Also reads ‘.data’ (binary), ‘.lines’ (text lines). And formatted lists: ‘.strings’, ‘.floats’, and ‘.ints’ (ex: “[1, 2, 3]”).

classmethod open_file(path: pocketutils.core.input_output.PathLike, mode: Union[pocketutils.core.input_output.OpenMode, str], *, mkdir: bool = False)

Opens a text file, always using utf8, optionally gzipped.

See also

pocketutils.core.input_output.OpenMode

classmethod write_lines(iterable: Iterable[Any], path: pocketutils.core.input_output.PathLike, mode: str = 'w') → int

Just writes an iterable line-by-line to a file, using ‘n’.

Makes the parent directory if needed. Checks that the iterable is a “true iterable” (not a string or bytes).

Returns

The number of lines written (the same as len(iterable) if iterable has a length)

Raises

  • FileExistsError – If the path exists and append is False

  • PathIsNotFileError – If append is True, and the path exists but is not a file

classmethod hash_hex(x: SupportsBytes, algorithm: str) → str

Returns the hex-encoded hash of the object (converted to bytes).

classmethod replace_in_file(path: pocketutils.core.input_output.PathLike, changes: Mapping[str, str]) → None

Uses regex.sub repeatedly to modify (AND REPLACE) a file’s content.

classmethod tmppath(path: Optional[pocketutils.core.input_output.PathLike] = None, **kwargs) → Generator[pathlib.Path, None, None]

Makes a temporary Path. Won’t create path but will delete it at the end. If path is None, will use tempfile.mkstemp.

classmethod tmpfile(path: Optional[pocketutils.core.input_output.PathLike] = None, *, spooled: bool = False, **kwargs) → Generator[pocketutils.core.input_output.Writeable, None, None]

Simple wrapper around tempfile functions. Wraps TemporaryFile, NamedTemporaryFile, and SpooledTemporaryFile.

classmethod check_expired(path: pocketutils.core.input_output.PathLike, max_sec: Union[datetime.timedelta, float], *, parent: Optional[pocketutils.core.input_output.PathLike] = None, warn_expired_fmt: str = '{path_rel} is {delta} out of date [{mod_rel}]', warn_unknown_fmt: str = '{path_rel} mod date is unknown [created: {create_rel}]', log: Optional[Callable[[str], Any]] = logger.warning) → Optional[bool]

Warns and returns True if path mod date is more than max_sec in the past. Returns None if it could not be determined.

The formatting strings can refer to any of these (will be empty if unknown):

  • path: Full path

  • name: File/dir name

  • path_rel: Path relative to self._dir, or full path if not under

  • now: formatted current datetime

  • [mod/create]_dt: Formatted mod/creation datetime

  • [mod/create]_rel: Mod/create datetime in terms of offset from now

  • [mod/create]_delta: Formatted timedelta from now

  • [mod/create]_delta_sec: Number of seconds from now (negative if now < mod/create dt)

Parameters

  • path – A specific path to check

  • max_sec – Max seconds, or a timedelta

  • parent – If provided, path_rel is relative to this directory (to simplify warnings)

  • warn_expired_fmt – Formatting string in terms of the variables listed above

  • warn_unknown_fmt – Formatting string in terms of the variables listed above

  • log – Log about problems

Returns

Whether it is expired, or None if it could not be determined