|
PySpark Plaso
Release 2019
A tool for distributed extraction of timestamps from various files using extractors adapted from the Plaso engine to Apache Spark.
|
Public Member Functions | |
| def | __init__ (self, package_dir=None) |
| def | make_package_dir (self) |
| def | register_package_dir (self) |
| def | pip_install (self, packages) |
| def | install (self, packages) |
Static Public Member Functions | |
| def | check_installed_package (package) |
Public Attributes | |
| package_dir | |
Python packages install class (not in a shell) to be executable by PySpark workers by
pyspark-driver:
sparkContext.addPyFile("custom_pip_package_dir.py")
pyspark-worker:
from custom_pip_package_dir import CustomPipPackageDir
cppd = CustomPipPackageDir()
cppd.install("future")
| def plaso.tarzan.lib.custom_pip_package_dir.CustomPipPackageDir.__init__ | ( | self, | |
package_dir = None |
|||
| ) |
Initializes the Python package install class. :param package_dir: a directory where to load packages from on the installation
|
static |
Check if the package is available. :param package: the package to check :return: True if the package is available, False otherwise
| def plaso.tarzan.lib.custom_pip_package_dir.CustomPipPackageDir.install | ( | self, | |
| packages | |||
| ) |
Install the packages by external call of pip if they are not already installed. :param packages: a list of packages to install :return: the output buffer of the pip
| def plaso.tarzan.lib.custom_pip_package_dir.CustomPipPackageDir.make_package_dir | ( | self | ) |
Create a custom module storage.
| def plaso.tarzan.lib.custom_pip_package_dir.CustomPipPackageDir.pip_install | ( | self, | |
| packages | |||
| ) |
Install the packages by external call of pip. :param packages: a list of packages to install :return: the output buffer of the pip
| def plaso.tarzan.lib.custom_pip_package_dir.CustomPipPackageDir.register_package_dir | ( | self | ) |
Register the package directory as a custom module storage.
| plaso.tarzan.lib.custom_pip_package_dir.CustomPipPackageDir.package_dir |
1.8.15